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ABSTRACT 

Thisse guidelines^ which summarize the content cf 
desegregation program evaluation studies, are organized into five 
chapters. The first is an "Introduction and Overview." Chapter 2 
specifies desegr^egation program variables whic^ are important in 
planning and ^lesigning evaluation studies. The structural and student 
variables that are required in the evaluation process are discussed. 
One important structural variable is racial balance, new and 
objective indicators of which are developed and discussed^ in chapter 
3. Chapiter 4 discusses currently available ef fiectiveness measures 
that school districts can use tc/ determin^e changes in students' 
achievement and racial attitude occurring as a consequence of 
desegregation. Chapter 5 discusses five major types of evaluation 
study designs which can be us^d to evaluate the effects" of 
desegregation programs. The advantages^ and disadvantages of each 
design approach are discussed in detail^ and evaluation issues which 
school districts must bear in' mind while planning and implem.entiag 
desegregation programs are identified* The nature and extent of the 
_jinf ormation provided by each design* approach, the faicilities and 
capabilities required to execute studies belonging to each design 
type, and iEportant cost factors a3sociated with each design type are 
also described. (Author/JM) / 
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"J- Chapter I ..^ - 

Introduction and Overview 

Ah important goal' of a racial desegregation program in- 
volving-changes in , the distribution, of .students^ to- instruc- 
tional units is to produce positive changes in students' 
attitudes and achievement. To achieve this goal, a school 
district first must formulate and execute a plan to alter * 
the distribution of students of different races to the Schpols 
and classrooms of the district.- Second/ the district must 
carry out an evaluation study to determine ' whether or not the 
desired structural changes in school or classropm composition 
haye been achieved, and if they have, to determine what 
changes in the cognitive and noncognitive behavior of^ stu- 
dents have occurred as a result of the structural alterations. 
These two steps are related, since the design of the evaluation 
study is strongly affected by the particular processes chosen 
by the district to achieve structural desegregation objectives, 
as well as by the nature and extent of the desired structural 
changes ref lected^in the desegregation objectives themselves. 

The particular desegregation objectives chosen by a 

district and the particular desegregation program adopted to 

implement these objectives will be, determined by many factors: 

« 

the district's assessment of the need for and the benefits to 



K " ■ 

be gained from desegregation; the social and political values 
of the district's population; the availability within the 
district of funds and other resources; and so. on. -In dealing 
with these complex and varied factors to formulate an appro- 
priate desegregation program, a district may easily--and 

understandably — lose sight of the need to plan an evaluation 
study concurrently ^with the formulation of the desegregation 
program itself. The possibility of inadequate evaluation 
study planning is heightened T)yf th^\many technical considera- 
tions, frequently unfamiliar to local district personnel, 
' which go into the design of an evaluation study. 

If a desegregation program is planned and executed 
without giving adequate . consideration to the study which will 
evaluate the effects of the program on students' development, 
evaluation of the program m^y yield ambiguous information 
or, in some, cases, may not yield any useful information at 
all*. Thus, if the evaluation study design does not gd hand 
in hand with desegregation program planning and implementa- 
tion, a district may find that it has expended both effort 
and resources to desegregate but is unable to determine What 
cognitive and noncognitive effects the desegregcition program 
has had on participating students. In the best interests of 
the children, a school district should seek to avoid this 
situation. As with any human endeavor^ and any educational 
program, good intentions do not guarantee success. If a 

-2- : ■ '/. 
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desegregation program fails to meet its objectives, well- 
designed and well-executed evaluation studies will reveal 
this fact, and serve t9 alert the district to the need ^to - 
take appropriate remedial action. 

The "purpose of these guidelines is to summarize the 
content of desegregation program evaluation studies so that 
districts canplan desegregation programs that will allow 
cognitive and' noncognitive changes in the behavior of partici 
pating students to be determined unambiguously. The remain- 
der of the guidelines is organized into four chapters, each 
dealing with a different element of the evaluation process/ 

Chapter II specifies desegregation program variables 
which are ' important in planning- and designing evaluation 
studies. The structural (e.g., environmental) and student 
variables that are required in the evaluation process are 
discussed, and minimum sets of s^tructural and student variabl 
required for evaluation studies ,are defined. ' "^--^^p 

One important structural variable is racial balance — 
the evenness with which minority students are distributed to 
schools and classrooms of a district. New, objective indica- 
tors of racial bal.ance are developed and discussed in Chapter 
III. These racial balance indicators are required to assess 
the effectiveness of desegregation programs in meeting stated 
objectives for- structural change, - as well as to analyze the 
effects of desegregation programs on students • attitudes and 

■ . . - 3 - 



achievement. 

Chapters IV and V deal directly with important issues in 

the design of the evaluation studies themselves. Chapter IV 

't- . ' 

discusses currently available effectiveness measures that 

■ ^> . . , . 

school districts can use to determine changes in students' 

achievement and racial attitudes occurring as a consequence 

of 'desegregation. 

Chapter V discusses five major types of evaluation study 

designs which can be used to evaluate thp effects of desegre- " 

gation programs. The advantages and disadvantages of each 

design approach are discussed in detail, and evaluation issues 

which school districts must bear in mind while planning and 

implementing desegregation programs are identified. The nature 



and extent of the informatio 



n provided by each design approach'. 



the facilities and capabilities required to execute studies 
belonging to each design type, and important cost factors, 
associated with each design type are also described. 

The State Education Department is prepared to support 
districts in the design and implementation of evaluation 
studies of desegregation p^rograms. The nature of the support 
which the Department is prepared to provide is described at 
appropriate places in the guidelines. 
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"Chapter II 
Defining the Desegregation Program 

The term "desegregation" has been used by school dis- 
tricts to refer tp many different processes. Consequently, 
a statement by a' district that it intends to desegregate does 
not by itself provide precise enoughs-information to enable 
formulation of a pjLan for ' evaluating the desegregation^ program 
Chapter II outlines the program elements which must be speci- 
fied by the district-in order to permit development of ah 
appropriate evaluation plan.. 

Key program elementjs which must be enumerater^^ include 

specific objectives with\ respect to restructuring the racial 

and social class composition of educational units , details of 

how these changes are systematicoliy to be brought about, and 

■ .\ ■ \ . . 

accurate descriptions of th^^.- predesegregation characteristics 

of tTie student Sody and the school environments which are to 

be affected ;by the desegregation program. 

A, Specif yingr\ Structural^ Desegregation Objectives 

In ordfer to develop an appropriate plan for evaluating 

program. effectiveness , the school district must be able to 

■ • ' \ ' ' ^ ' ■ ^ - ■ 

answer* accurately the following questions' regarding its deseg- 

regation objectives for the structure of educational units, 
1, What is the nature of the desired change in student 
body composition ? The types of change most frequently sought 



by districtG undergoing desegregation are changes in the racial 
balance, racial heterogeneity and, tb'a lesser degree, social 
class balance and i^ocial class heterogeneity of schools and 
classrooms. ^ 

o Racial balance refers to ^the evenness with which minority 
students are distributed to educational units. Is the program . ' 
designed to alter the racial, balance of classes within schools 
and/or the raciaL^balance of schools within districts? If so, 
what iS| the magnitude of the change desired? — 

Racial heterogeneity refers to the actual proportion of 
minority^ students in a class, grade, or school; (Whereas 
racial ^alance refers to the evenness or randomness of distri- 
bution^ heterogeneity refers simply to thie proportion of stu- 
dents available to be distributed.) . Is the desegregation 
program, designed to change the racial heterogeneity of ^schools 

or classrooms? If so, what is the magnitude of the desired 

■ . . ■ ^ ■ 

change? . • K ^ ' / ' ' 

: Social class balance and social class heterogeneity are 

analagous to racial balance and racial heterogeneity, but 

refer to the socioeconomic (SES) characteristics of educational 

' environments rather* than to their racial characteristics . 

is the alteration of the social class balance or heterogeneity 

of schools dr. classrooms an objective of the desegregation 

program? If so, what is the nature and extent 'of the change 

desired? . ' 
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2,. How ei^tensiv'e will the desegregation be ? 

a, ^ How many grades are to be involved: one grade, 
^several grades, all grades? . 

*'b. How, many schools are to be involved: one sbljool, 
several schools, all schools? ^ 

c, - What proportion of studen^ts in the affecl:ed grades* 
ai*id schools are to be ^involved? ^ \ 
B. Defining Desegregation Processes 

A wide variety of processes have been employed .by school 
districts, to meet their desegregation objectives,. Adequate 
description of any desegregation program therefore requires 
that the district specify which method(s) will be used to ^ 
bring , about ' the desired changes in student body composition, 
and how- it (they) will be implemented (see Table II-l for / 
, examples).^ If more than one mjsthod . or more than' one mode of 
imp!^.ementatibn is to be; used, the district must specify which 
methods and modes wi"ll be used for which students. 
■ Describing ^Student Characteristics and Predesegregation 
Environments - . 

There [are two iinportant reasons for objectively describing 
the characteristics of students and of educational environments 
prior to desegregation. First, if it is the" objective of the 
desegregation program to alter thjS racial or social class com- 
position, of schools, grades, or classes, an accurate record of. 
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these educational environments must be obtained prior to 
implementation of the progrcin, so that there will be adequate 
baseline data to permit determination of whether or not the 
desired changes have occurred. • 

Second, an accurate description of student and environ- 
mental characteristics^ is needed so-- that the research design 
which is developed (see Chapter V) can take into account as 
many as possible of the variables or factors which might 
influence the outcomes of the evaluation study., 

The minimal information which must be known for each ' 
student are his. school, grade, class assignment^ race, and 
sex. Data on SES background, IQ, 'and prior academic achieve- 
ment are also desirable (but will be useful only if the 
measures or indices used are comparable for all students)-. 
This set of data for each student will enable the district to 
describe accurately the nature and distribution of the student 
body by grade and by school. . ^ 

Using this information about students, districts, can 
'characterize classes, grades, and schools with respect to a 
number of environmental variables. The minimum set. of en- 
vironmental variables which should be indexed (see Chapter III) 

consists of classroom racial balance, classroom racial' 

. • ■' ' 

heterogeneity, school racial balance, and school racial 
heterogeneity, for every grade in which the desegregation 
program will operate. / 
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■ If possible, the SPS balance and/or heterogeneity of 
classes and schools should also be assessed (this is a definite 
requirement whenever a change" in SES' balance or heterogeneity / 
is a specific program objective) , as .should the level and 
variability .of IQ and achievement. Moreover, since teacher ' 
characteristics such as race^ amount of experience^ and verbal 

ability may^ influence the outcomes of desegregation, it' is 

'/ 

appropriate to index them. To be of utility in the progr;^ 
evaluation, these variables should be assessed in comparable 
ways for all classes and -^for all schools in the grade levels 
affected by desegregation. 

The environmental characteristics, or variables, that are 
indexed will' be used in what is essentially a two-stage eval- 
uation process. In the first stage (described in Chapter III), 
the. values of these variables prior to and following desegre- 
gation will be compared. Thus during the first stage of the 
eyaluation process, changes in the values of the environmental 
variables will themselves be analyzed: These changes constitute 
the effectiveness measures (dependent data) which will deter- 
mine the degree of success of the program in accomplishing 
its objectives for structural change. 

If analysis of the environmental indices shows that the 
desired structural changes have occurred, the district may 
proceed with the second phase of the evaluation process\/ In the 
second stage (described in Chapter V) , these same envirbnmental 



characterii^'tics become the classif icatory (independent or 
control) variables in terms of which the cognitive and non- 
cognitive effects of the desegregation program are analyzed.*" 

However, if the analysis-of changes in the environmental 
indices shows that the variables have not been altered in the 
desired way, there is no rational basis for the district to 
pi^oceed to assess any cognitive or noncognitive program 
effects, since the structural changes which were supposed to 
define the desegregation program have not taken place. 



Even if the desired change in student body composition has 
occurred, there will be instances in which i<t will not ^be- 
worthwhile to assess cognitive and noncognitive program effectr. 
Such instances are those where only a handful of minority 
students* are involved in the program and where the progr'am may 
be categorized as a "demonstration pro ject . " An example would 
be the case where a small number of urban minority students, 
(e.^g., 30 or fewer) drawn from several grade levels are bused 
to an all white school .in their own or a neighboring district. 
Since the program is apt to be permeated by an air of novelty, 
since the bussed students are apt to be a highly se-lect group, 
and, most importantly, because there are so few minority stu- 
dents involved at any grade level, it will usually not be 
worth the' expenditure of district resources to undertake 'the 
Vork which would be required to assess cognitive and noncognitive 
outcomes . * 
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Chapter III 
Racial Balance Indicators 
The minimum set of environmental variables required to 
describe schooT environments for the purpose of performing ' 
evaluation studies consists of school racial bv^lance, class- 
room racial balance, school racial heterogeneity and .class- 
room racial heterogeneity (see Chapter II) . School and 
classroom racial heterogeneity--the proportions o£ minority 
students in schools and classrooms' — are easy to calc^alate if 
sufficient information about student distributions has been 
collected. School ahd classroom balance — the evenness with ' 
which \inority students are distributed to schools and class- 
rooms — are much more difficult to characterize objectively. 
Sections A and B of this chapter deal with the construction 
of objective indicators of racial balance, and with the 
presentation of racial balance information to facilitate 
decisionrmaking. Section C considers practical problems of 
data collection, data processing and the use' of racial bal- 
ance information to monitor and evaluate desegregation programs 



- 12 - 



ERIC 



^ . . ' ■ 

A. The Need for Objective Indicators of Racial Ba\lanc;e 

A school district contemplating a, program which will 
alter the distribut^ion of minority group children in the 

schools and classroopis of the district--that is, one which 

j • ■ . ■ 

will affect the pattern of racial balance--must be able to 
characterize this distribution in quantitative and objective 
terms. Without objective indicators^" the' extent of/ 
racial balance within the classrooms and schools of a dis- 
trict, and the changes in racial balance which occur when 
racial desegregation programs are implemented, become mat- 
ters of individual judgement, providing (unfortunately) 
fuel for an extended debate which can reach no constructive \ 
conclusion. Approaching problems of racial balance without 
objective indicators is like examining a business which keeps 
no financial records: the financial health of the busines^'^ 
and its future prospects become anybody's (or, in, this analogy, 
everybody's)' guess. - / 

In these 'guidelines, school and classroom racial balance 
have b.'iien given precise meanings. The definitions of; school 
and , classroom balance, which appear on page 19, directly ' re- ■ 
late racial balance to the- extent to .which a distribution of 
s'tudents to schools and classrooms is ' nondiscriminatory , and 
are consistent with the intent of desegregation legislation. 
Furthermore, the ioidicators which follow from these definitions 
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are entirely descriptive; they characterize student ..distri- 
butions, but say nothing about the consequences to students 
of school and classroom environments of varying racial bal- 
ance. Relating cognitive and noncognitive student- behavior 
to different levels of racial balance (and other variables) 
is an entirely separate matter, and is left to evaluation 
studies of the types considered in Chapter V. 

Objective indicators of racial balance can be of great 
use to local administrators and decision-makers in the for- 
mulation and implementation of desegregation programs. 
Specifically, such indicators would serve: 

• To identify unambiguously instances of racial imbal- 
ance which exist within the . district ; 

• ^ To permit a^lternative desegregation programs to be 

sensibly compared. The structural effects of different 
proposed desegregation programs could be assessed by 
calculating the changes in -^the values of the racial 
balance indicators which each alternative program is 
.. expected to produce . If the costs of altof'native 
programs were estimated, cost-effectiveness compari- 
\ sons could also be carried out; 

\ To permit desegregation programs to be monitored and 

. evaluated. If the values of racial balance indica- 
\ tors were obtained at the inception and the comple- 
tion of a desegregation program, the extent/ and 
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nature of the racial balance achieved as , a result of 
the program could be determined unambiguously. In 
instances where .desegregation programs extended over 
severa^l years, changes in the values of the racial balance 
indicators could be used to monitor progress annually 
or semiannually. 

Establishing the need for quantitative, objective 
indicators of racial balance is considerably easier than 
generating the indicators themselves^ The%iscussion in 
this section of the guidelines will show that care must be 
exercised in the definition of racial balance indicators if 
real objectivity and a high degree of practical utility -are 
to.be attained. Appropriate racial balance indicators must 
be sensitive only to those properties of student distributions 
which are under the control of local administrators r and 
must not be affected by factors such as district student 
population size; and district racial heterogeneity which 

local administrators cannot cont:.r;i. \ ■ . 

. ■ ' / 

-The matter of identifying those properties of student 
distributions to which racial balance indicators should not 
be sensitive is basic to the development of useful objective 
indicators. From the viewpoint of developing racial balance 
indicators, the most important "uncontrollable" differences ' 
are these: 

- 15 - 
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• Size of the dibcrict student population ; 

• Racial heterogeneity of the district student popula - 
tion . This factor includes not only the number of. 
student minority groups in the district (black, 
Puerto Rican, Indian, oriental, etc.) but also the 
proportion (i.e., the fractiori) of students f rom Neach 
minority group ^at each grade (or age) level in the 

« district student population. The proportion of 
minority group students at each grade (or age) level 
may vary widely from grade to grade as well as be-" 
• -tween districts. In fact, . "minority" students may 
' constitute the "majority " at -some grade levels in 
certain districts. 

• Year-to-year changes in student population size and 
racial heterogeneit y. Changes will occur because of 
the migration of families with children into and out 
of the district, or because of redistricting. 

In effect, these factors ' represent a set of constraints 
imposed on local administrators. Local administrators must 
operate within these constraints to achieve ; racial balancq 
in the district's schools. Therefore, a condition on the 
following discussion will be that appropriate definitions of 
racial balance as well as appropriate formulations of racial 
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balance indicators must be unaffected by these constraints.* 

a 

The . early identification of these constraints will s'erve to 
guide the remainder of .the discussion in this chapter. 

For a district student population of a given size 
and racial heterogeneity, the process by ^which students 
arrive at particular classrooms begins with the enrollment 
of the students in the various schools of a district: The 
administrators in each school then assign students enrolled 
in that school ^to classrooms or other instructional units. 
Investigations of racial balance in schools and districts 
indicate that the appearance of students in classrooms can' 
be accurately represented by such a two-step process, and 
furthermore that the first step* (the enrollment of students 
in schools) can be represented independently of the second 

(the assignment of students to classrooms within schools). 
The fact that students are not assigned, directly to classrooms 
on a district-wide basis suggests that, two "kinds" of racial 
balance" must be considered for each, district : school balance 

(characterizing the outcomes of the enrollment of the dis- 
trict's students in different district schoqls) , and class- 
room balance (characterizing the outcomes of the assignment 



This condition should not be interpreted to mean that the 
constraints cannot .or,' in the best interests of the children, 
should not be changed. Rather, the condition reflects the 
Judgement that alterations in the constraints transcend the 
authority of local administrators, and that in any given schoo. 
year administrators can only operate within the constraints to 
achieve the best possible results. 
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of students in each school to ' classrooms ) ; and that the 
indicator for each "kind** of racial balance should be 
' independent of the other, 

This last point--that school and classroom balance " 
are independent quantities — requires some amplif ication>;^^,\^ 
Just as district administrators do not control the racial . 
heterogeneity of a district's student populatioii, so indi- 
vidua! school administrators .do not control the racial 
heterogeneity of the student population enrolled in a 
particular school.* School administrators must "work with 
what they have." Thus an indicator of classroom balance 
should not be affected by ^t'he properties of the studenjb 
populations which school administrators assign to classrooms 
School balance and classroom balance are significantly dif- 
ferent aspects of district racial balance, and each must be 
considered separately in assessing the status of . racial 
balance in the district as a whole. 

School balance and classroom balance can be defined 
in operational terms as follows: 

— '\: '/ - ' 

* - /" 

School racial heterogeneity is yaf fected by any procedure 

which governs the enrollment of students in schools, i.e., 
school racial heterogeneity depends on school racial balance 
However classroom balance shoul.d be independent of school 
racial heterogeneity just as school balance should be inde- 
pendent of district racial heterogeneity. 
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• School balance is the extent to which "the pro- 
portion of students from a particular minority group at each 
grade level in every school approaches the proportion of 
students from that minority groun at that level in the dis- 
trict as a whole. Note that the concept of school racial 
balance is applied separately to each minority group; for £ 
each minority group, the proportions referred to are cal- 
culated with. respect to total populations of the school and 
the district, regardless of how many other minority groups 
are represented in^ those total populations; 

• Classroom balance is the within-school counter- 
part of school balance. Classroom balance is the extent to 
which the proportion of students from a particular - group at 
each grade level in everj' class approaches the proportion of 
students from that minority group at each grade level in the 
school as a whole. Again, the concept of classroom balance 
is applied separately to each .minority group; for each 
minority group, the proportiqns referred tc> are calculated 
with respect to the total populations of the .classroom and 
the school, regardless of how many other minority groups are 
represented in those total populations. 

It is the purpose of Section B of this chapter to obtain 
quantitative indicators of school and classroom balance and 
to prct>po£e ways of presenting racial balance data to adminis- 
trators, school boards and other decision-makers in school 



districts. However, the operating definitions which appear 
/above raise several points which* require further discussion 
before prpceeding with the development of the indicators. 

First, the definitions of both school and classroom 
balance treat each minority group separately because, although 
segregation wiir certainly alter the behavior of minority 
group students that are subjected to it, there is no evidence: 
.th^--Lj[nembers of different minority^ groups will react in the 
same 'way to a segregated environment. Aggregated racial balance 
indicators can be developed which determine school^, and classroom 
balance for several minoriity. groups combined. However, it v^ould 
be impossible to- tell from such- coiribined indicators whether stu- 
dents from one minority group were more or less evenly distri- ' 
buted than students from another minority group. Furthermore, . 

■ 4\ . ^ .... 

each minority group will wai^t to know how its members are distri- 
buted in the schools and classrooms of the district. Therefore 
it appears to be advisable to treat s'tudents from eaqh minority 
group separately. In the event that a district has only a few 
students from a particular minority group height Indians, forb 
■ example) / it would^ .probably be best to handle them on an indivi- 
dual basis. 'The calculated values of the racial "^balance indica- 
-tors developed subsequently in this chapter behav^ erratically 
when only a few minority group students are' being distr-ibuted. 



Second, £h,e definitions of both school and classroom • 
balance treat each' grade separately because, as discussed 
earlier in this section, , district racial heterogeneity ^ may 
vary widely from grade to ^rade within a district, and this 
is considered to be an "uncontrollable" factor to which the 
vaxues of balance . indicators should be insensitive. However, 
the fact that each grade is, treated separately v does not mean 
,that balance indica^tors f or each grade cannot be combined in 
an attempt tp summarize in one number the status of school or 
classroom balance throughout the district. 

The problem with such summary numbers '-'lies in thjeir 

interpretation. A great deal of information is lost in 

» 

combining balance indicators for several grades, particularly 
when racial heterogeneity or racial balance or both vary 
widely^' from grade to grade. Under such circumstances , using 
summaries exclusively can have the effect of obscuring the 
existence of racial imbalance in certain grades or achools of 
a district, and f-esult, for example, in a poor choice among 
alternative desegregation plans. Consequently; district- 
wide summajries should be used with caution, and ^never to the 
exclusion of grade-by-grade and school-by-*school .balance 
indicators. v - , 

Finally, there may appear to be problems in applying 
the definitions of school or classroom balance if the schools 
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within a district depart from the traditional classroom- 
within-grade instructional pattern in which the same students 
are always gjrouped together with the same teacher to receive 
instruction., In practice, the definitions of school and . 
classroom balance-^-and the balance indicators which follow 
from them--can be easily adapted to accommodate different 
instructional patterns such as ungraded instruction and the 
periodic regrouping of. students in different instructional 
units . ,.if isome snhcols within a district arc ungraded, the 
ages rather than the grade levels of students can be used as 
a basis for determining balance indicators. If the classroom 
to which a student is assigned is an administrative unit 
(or "home room") thathas no relationship to- the classes in 
.which instruction is received, classroom balance as defined 
above- may be replaced with "instructional unit" balance, and 
each instructional class* within each classroom hour considered 
to be a self-contained classroom. This situation would in- 
crease somewhat the amount of data required to determine the 
analog of classroom balance, but would not significantly 
complicate the computation of the corresponding racial balance 
indicator which is carried out on a digital computer. If the 
task of data collection became burdensome, it could be reduced. 



* Lunqh, assemJDly and homeroom periods would be- omitted. 
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Sampling procedure^ have been devised from which the degree of 
"instructional unit" balance in the student population (f or ^. - 
different schools or for the whole district) could bq inferred 



; i 

/ . - 




* — A detailed description of the sampling approach is -outside 
the scope of these guidelines. 



. V ■ ■ ■ ■ 

B. A Precise Way to Characterize Changes in School Environ - 
ments Resultincr From Desegregation Programs 
1. Preliminaries , The principal purpose of this section 

of the guidelines is to develop valid, quantitative indicators 

of school and classroom balance that conform to the opera- 

'^tionai definitions and are insensitive to the "uncontrollable** 

■ f ■ . 

factors discussed in the previous section of this chapter. 

These indicators, together with school and classroom racial 
hete'fogerieities form the minimum sqt of variables used to de- 
scribe school environments. The development in this section 
is descriptive rather than rigorous, and employs examples of , 
simple distributions of students to classrooms or schools to 
illustrate concepts and problems. 

The student distributions used as examples in this 
section will be presented as shown in Figure III-l. Figure 
III--1 shows a distribution of students from minority group m 
(blacks, for instance) ..in the classrooms (eight in this 
example) of the fifth grade in a peirticular district school 

(labeled s) . Note that two kinds of numbers appear in the 

- 1 

\ ■ 

figure: Those within the heavy rectangle; and those at the 
edges, or margins, of the rectangle.. The numbers within the 
rectangle constitute the distribution of. students from 
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minority group m (top row) , and of students not trom minor- 

J- 

ity group m (denoted as r; bottom row) , to the eight, class- 
rooms of the grade. The marginal numbers are various totals 

which will be useful at ^certain stages of the discussion; 

■<«> **• 

they are defined directly on the figure i 

A similar array can be used to specify the distri- 
bution of m-students (from minority group m) and r-students 
(students from all other groups - the "remainder") in a parti- 
cular grade to the different schools of a district. In 
this case, the columns of the array would denote different 
schools (rather than classrooms) . The marginal totals at 
the right would denote numbers of m-students and r-students 
in the particular grade in the district, the marginal totals 
at the bottom would denote the total number of students in 
the particular grade in each school, and the grand total at 
the lower right corner would give the total number of stur- 
dents in the district . at the particular grade level being 
examined. 

2 . Difficulties with simple indicators of racial bal - 
ance . In order to develop suitable racial balance indica- 
^tqrs, a quantitative way must be found to characterize student 
distributions. The most direct and simplest way to develop 
an indicator is to work directly with the percentages of minor 
ity students (m-students) in each class or school. These per-- 
centages can be manipulated and displayed in various ways to 



try to describe the extent of school or classroom balance exist- 
ing in\the schools or classrooms of a district. 

Indicators constructed from percentages of minority 
studeij^^s have been used frequently to measure racial balance. 

either the classrooms in a school "or the schools in a dis- 
trict.,, the following quantities can be defined (as usual, for 
a particular minority group and a particular grade) : 

• minimum percent . The smallest percentage of 
minority students found in any classroom (or school); 

• maximum percent . The largest percentage of 
minority students found in any classroom (or school) ; 

• percent range . The difference between the 
maximum and minimum percents ; 

• average deviation of percents . Suppose p # p^, 
p^... are the percents of minority students in different classes 
(or schools), p is the percent of -minority students in ^he school 
(or the district) , and n is the number of classes (or schools) . 

/ Then the average deviation of percents (AD) is defined as 

AD = - - 
.where thfe vertical lines around |p^"p|^ indicate 
that the" positive version of the two possible differences.. 



t 

Cj 

(Pj_-'P) and (p--p.j_) , should be chosen in order to jiiajsie each 
term in the AD positive.* ^ 

To see why these quantities should not be used as 
indicators of racial balance^ consider the four examples 
of classroom distributions shown in Figure III-2 for a partic- 
ular .minority, group. In the first distribution, classes 1 
through 6 are perfectly balanced (the proportion of minority 
students in these classes is the same as the proportion of mi- 
nority students at that grade level in the school) , while classes 
7 and 8 are maximally unbalanced. In the second distribution, 
all classes are maximally unbalanced. Note that the maximum 

percent, minimum percent and percent range are the same for 

these two distributions even though the classroom balance 

pattern is. quite different. . 

Hence maximum and minimum percents and percent ranges 

are poox* indicators of racial balance. ; The reason they are 
-poor indicators is because they only characterize the extremes 

bf the classroom' distribution, and many different 'distributions, 

differing greatly in their patterns of racial balance, can have 

the same extremes. 



The AD can also be written 

■' • 

J n , . 

EIp -p| 

-where the S indicates the addition shown explicitly in the first equation. 
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The average deviation (AD) looks like a stronger choice 
for an indicator of racial balance. The AD is much smaller for 
the first distribution than it is for the .second, reflecting the 
first distribution's greater extent of classroom balance. The 
difficulty with using the AD as a racial balance indicator 
is illustrated by the second, third, and fourth distributions. 
All three show the same pattern of classroom balance, spe- 
cifically complete imbalance, yet the AD decreases in vadue 
from 50% for the,, second distribution to 22% for the fourth 
distribution. This decline in the AD is a consequence of 
its dependence on the different racial heterogeneities 
(shown in the first column of the figure) of the student 
populations in the second, third and fourth distributions. 
Therefore the average deviation is* a poot indicator of racial 
balance because it is affected by 'one of the factors consid- 
ered to be "uncontrollable" in the previous section of this 
chapter. 

Simple indicators racial balance suffer from 
one or both of the faults illustrated by the above examples: 
either the indicator does not characterize student distribu- 
tions precisely enough to allow different patterns of racial 
balance to be distinguished from one another; or -part of the 
variation in the indicator is attributable to one or more of 
the "uncontrollable" factors identified in the previous sec- 
tion? or both. Therefore the status of racial balance in a 



school' district can be misrepresented if these simple indica- 
tors are used to measure school or classroom balance. If 
decisions concerning racial desegregation programs are^" based 
on the use of simple indicators, such decisions could produce 
consequences inconsistent with the objectives, of racia-l deseg- 
regation. 

3. A new indicator of racial balance . 

a . . A different approach . The previous discussion 
shows that a different way to characterize student distributions 
(other than by such simple properties as minimum percents, maxi- 
mum percents, and so on) must be found if racial balance indica- 
tors that meet the requirements of Section A of this chapter are 
to be developed. The- property that suitably characterizes each 
student distribution, and _that leads to acceptable racial balance! 
indicators , is the probability tha:t a particular distribution 
will occur if students are -assigned at random (that is, without 
knowledge of the racial groups to which the students belong) to 
schools or to classrooms. The procedure for calculating this 
probability is well known.* Precisely the same procedure is 
used to calculate the probabilities (or "odds") of different 
card combinations in bridge or poker. 



See any text on the theory of probability, e.g., W. Feller, 
■• An Introduction to Probability Theory and I ts Applications, 
vol. I. John Wiley and Sons, New York, 1950. 
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A simple example will serve to illustrate how a 
■distribution is characterized by its probability of occurrence, 
and to illustrate how this probability reflects the degree of 
racial balance inherent in different distributions. Suppose 
we wish to distribute a student population consisting of 12 
students, 6 students from minority group m and 6 students 
from the "majority" group (labeled r)., to two classrooms. 
Suppose also that each classroom accommodates 6 students 
(that iS; the class size is fixed at 6 students). In this 
case, there are seven possible distributions • These distri- 
butions are shown in Figure III-3, along with the probability 
that ^each will occur if the 12 students are distributed, six 
to each classroom, at random. 

The interpretation of these probabilities is 
quite simple. If the 12 students were distributed to the 
two classes without any consideration being given to their 
race, there is only one chance in a thousand (0.1% probability) 
that distribution l and distribution 7 would occur.* In 
contrast, there are just over four chances in ten (43.3% 
probability)' 'that distribution 4 would occur. The proba- 
bilities of occurrence of the other distributions fall between 
these two limits, as shown in the figure. 

The probability of occurrence of the least probable distribu- 
tion falls off very rapidly as the student population increases. 
' For example, for 16 m-students, 16 r-students and two class- 
rooms of equal capacity, this probability is approximately 

10*"^, or 0,0000001%. 

i" 
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Fig.. III-3. Distribution of 6 m-students (from minority group m) 
and 6 r~students to two classes, six students to each class. 
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Distribution 4, the most probable distribution, 
corresponds to maximum classroom balance: the proportion of 
minority students in each class equals the proportion of minority 
students in the population of 1:2 students being assigned. Distri- 
butions 1 and 7^ the least probable distributions, correspond 
to minimum classroom balantie:* the proportions of m-students, 
in both classes' are as far as they can be from the proportion ^ 
of m-students in the population of 12 "students being assigned. 
Distributions 3 and 5 are iess balanced than- distribution 4 
but more' bala^nced than distributions 2 and 6, and so on. j 

Thus the probability\.of occurrence of a particular 
classroom distribution is directly related not only to the i 
operational definition of classroom balance given in Section .. 
A of this chapter, but also to the degree to which the intent 
of racial desegregation — that assignments to classrooms be 
made without taking the race of the students into considera- 
tion — has been fulfilled. If student distributions of^low 
probability are encountered (i.e. , large imbalance) , ,a high 
likelihood exists that the race of the students was a factor 
in bringing about those distributions. The lower the proba- 
bility associated v;ith a distribution , t^he higher the . 
imbalance, and the hiqher the likelihood that the assignment 
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of students was influenced by knowledge of race.* These state- 
ments apply not only to distributions cf students to class- cr> 
rooms withix^ a. schobl but to distributions of students to 
schools within a district as. well. 

The probability of occurrence of a student distribu- 
.tioQ, unlike the simple indicators discussed earlier , is so 
intiniately related to the extent of racial Jbalance in that 
distribution,^ that it is tempting to use the probability it- 
self . as a quantitative' indicator of racial balance. Unfor- 
tunately, the numerical value of the probability- is affected 
by "uncontrollable" factors, such as the size and racial 
heterogeneity of. the^ student population being assigned, to 
which a suitable balance indicator must be insensitive.** 
Therefore, a measure, or function, of the probability must' be 
found from which school and classroom balance indicators 
satisfying all the requirements discussed in Section A of 
this chapter can be constructed. 



*• Or influenced by knowledge of some characteristic which is > 
strongly correlated with race. For example, if the black stu- ^' 
dents in a particular district live 'in a ghetto area, then^' the 
placa^f 2;;esidenc^_af-'a student is strongly correlated wi^h the 
studeritls^r^tce'T'^ In this case, assigning students to schools on 
the basis of where they reside in the district would be equiva- 
lent to assigning students ' to'* schools on the bases of their race 

** These properties of the probability are discussed in the 
appendix . 



b. Constructing the indicators . In order to formu-- 
late suitable indicators, a certain amount of mathematical, 
maneuvering is necess'^ary, beginning with an examination of the 
formula used to calculate the probability that a particu- 
lar student distribution will occur. It is not necessary 
£o know specifically how an indicator is obtained in order 
to usd it to calculate school or classroom balance, an^ more 
than it is necessary to. know how-to calculate the odds \in 
poker in. order to use the results in playing the game, ^ 



Consequently, the technical discussion leading to the for- 
mulation of the indicators described below has been placed 
in an appendix to these guidelines, and the discussion 
in this section has been limited to a description of how 
the indicators are constructed and interpreted- The for- 
mulas required to calculate the indicators arfd other 
related quantities appear in footno^tes. 

The data required to determine numerical values 
of the school balance indicator for a particular grade (or the 
equivalent) are the number of students at that grade level 
from each group (minority and "majority") in each school 
within the district. The data required to determine, numeri- 
cal varlues of the classroom balance indicator for a particular 
grade (or the equivalent) and school are the number of students 
from each group (minority and "majority") in each classroom 
(cr the equivalent) of the grade. (See Figure III-l for an 
example of how classroom data might look.. 



For each minority group and grade level (or the 
equivalent) , indicators of school balance within the district 
and classroom balance within each school- are separately con- 
structed. However, since the process of distributing students 
to schools and the process of distributing students to class- 
rooms within schools have a common. mathematical representation, 
the f orm (although not the meaning) of the school and classroom 
balance indicators is the same. 

• School balanc^ indicator. The school balance 



indicator for a particular minority group m and a particular 



grade g will be denoted as 



mq 



The .school balance indicator 



is expressed as a percent and is constructed as follows:* 



B^^ = 100 X 
mg 



(T ) - T 

mg max mg 



mg max 



ma min 



The quantity T which appears in this formula Is defined as follows: 



mg 



log N-1 
2 N 



where : 



N ^ total number of students in grade g in the district (fixed); 
^m * total number of students from minority group m in grade g in the district 
(fixed); 

= N - = total number of students from all other groups (the "remainder") in 

grade g in the district (fixed); 
=. total number of students in grade g in school s (fixed); 
^ms ^ number of students in grade g and school s from minority group m (variable) ; 
^ra ~ \s r^ii^^er of students in grade g i:i school s from all other groups 

(variable) ; 

S = th^ number of schools in the district (fixed). 
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The^ quantity (T ) ' is the value of T which corres- 
^ mg max mg 

ponds to the least probable distribution of students from 
minority group m and grade, g to the schools of a district 

(minimum school balance). For the least probabl^distri- 

/ 

bution, T takes on its maximum possible value^. The quan- 

mg^ ' . ^ 

tity (T ) . is the value of T* which corresponds to the 
mg'min mg y ^ 

most probable distribution of students from minority group m 

/ 

and grade g to the schools/of a district (maximum school 

balance). For the most /probable distribution, T takes on 

/ mg 

its minimum possible vAlue. .The quantity formiri'g the deno- 

/' •■ ' . 

minator of B_ , (T / ) - (T )_. , is just the range of 
^9 [ mg max mg minj 2_ 

possible values which T - can take on for different distri- 

mg 

butions of students from minority group m and grade g to^the 

schools of a district. 

The quantity [(T^^)^^^ "•'(T^^)^^„ can be compared to 
^ L nig nisx mg minJ 

a path to school balance. 'The path runs from minimum school 

balance, ^"^mg^max' maximum school balance, C^^ig^niin* 

T is a point on this path, and |(T^^)^^^^ is the 

mg ^ L ^5 max mgj 

distance which the district has traveled along the path to 

maximxim school balance. B_ , then, can be interpreted as 

mg 

the extent of school balance achieved by a district (for 
students from minority group m and grade g) relative to the 
extent of balance that the district could achieve, given the 
size and racial heterogeneity of the student population en- 
rolled in grade g, and assuming fixed school capacities. 
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• Classroom balance indicator > The classroom 

balance indicator for a particular minority group m, grade 

g, and school s will be denoted as b ^. The classroom 
^ . mgs 

balance .indicator is expressed as a percent and is ,cpn- 
structed as follows:* 



mgs 



^ Sags ^ max " ^- 

TtZZ)ZZ - it 



mgs; 



mgs max 



mgs ^min 



The quantity t which appears In this formula is defined as follows; 



^ms^^^^mi ^^^^.rs^^^.^s ^^^V^^.^c^ "S^^c^^^.^c * "rc^°^."rc^ 



where: 



= total nuinber of students in grade g in school s (fixed); 
Nj^^ = number of students in grade g and school s from minority group m (fixed) ; 
N^g ~ " ^nis " nuinber of students in grade g in school s from all othgr groups (f^xed); 
n^ =■ total number of students in classroom c of grade g and school s (fixed) ; 
n^^^ = number of students in classroom c of grade g and school s from minority group m 
(variable) 

n^^ = n^ - n^^^ = number of students in classroom c of grade g and school s from all 

other groups (variable); 
C the number of classrooms in grade g and school s (fixed) . 



- 39 - 



ERIC 



The indicator of classroom balance, b ' * can be 

. -rngs 

interpreted in precisely the same waiy for student assignments 

to the classrooms of a particular grade and school as the 

school- balance indicator, B , was interpreted for student 

mg ^ 

assignments to schools. The indicator b is the extent of 

mgs 

classroom balance achieved by school s (for students from 
minority group m and grade g) relative to the extent of balance, 
that the school could achieve, given the size and racial 
heterogeneity of the student population to be assigned to the 
classrooms in grade g in the school ^ and assuming fixed class- 
room capacities. The quantities (b ) and (b ) , cor- 
^ ^ , mgs'max m.gs min 

respond respectively to the least and most probable distribu- 
tion of m-students to the classrooms of grade g in school s, 
and so on. 

The construction of maximally balanced and unbalanced 

school and classroom distributions and the calculations of 

B and b from school and classroom data are intended to 
itig mgs 

be performed on a digital computer. The formulas for T^^; and 

t aiven above are in a form suitable for machine calcula- 
mgs - 

tion. Rivierside Research Institute staff have performed 
these calculations for over 20 simulations of student distribu- 
tions to schools and to classrooms. The results of these 
calculations, some of which are used as examples later in this 

section, support the- following conclusions: B and b are 

mg mgs 



- 40 - 



ERIC 



indicators of school and classroom' balance that are consis- 
tent with the operational definitions of school and classroom 
balance introduced earlier; they are independent of one 
another; and they are^not sensitive to factors such as stu- 
dent population size and racial heterogeneity. 

Although both B and b are quantitative indicators, 
^ mg mgs ^ 

the calculated values of these indicators must be interpreted 

with care. The range of values of the indicators B and 

mg 

b constitute ordinal measurement scales • * Ordinal scales 
mgs 

can only be used to order observed phenomena systematically 
(in this case, student distributions to schools and class- 
rooms) • For example, if two distributions have the same 
value of B^g, they can be said to represent the same extent 
of racial balance. Furthermore, a distribution for which 

B = 50% can be said to show a greater extent of school 
mg 

balance than a distribution for which B =25%. However, 

mg 

it -is not correct to conclude that the distribution for which 

B = 50% reflects twice as much school balance as the dis- 
mg 

tribution foi: which B = 25%; that is, ratios (and, for that 
matter, sums and differences) of values of B^^ for different 
distributions have no real significance. 

* For a discussion of measurement scales, see, for example, 
S. Siegal, , Nonparametric Statistics for the Behavioral 
Sciences , chapter 3. McGraw-Hill Book Co=, New York, 1957. 
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These limitations should be borne in mind when several 
school or classroom balance indicators are combined to sum- 
marize the status of school or classroom balance throughout 
a district. Combinations of balance indicators also give 
rise to ordinal scales, and only relative, significance 
should be attached to the particular values which these 
combinations may take on. 

Particular values of school and classroom indicators may 
acquire increased significance as a consequence of investiga- 
tions into the effects of different segregated environments 
on the cognitive and non-cognitive behavior of children from 
different minority and "majority" groups. In this event, school 
and classroom indicators can be adjusted to accommodate the re- 
sults of such investigations. For example, preliminary data 
indicate that when only one or two black students are in a 
classroom, they tend to be more alienated than when the number* 
of black students^ in the classroom is somewhat larger.* For 
this reason a school having only a few black students in a par- 
ticular grade may wish to maintain a degree of classroom im- 
balance in order to have no less than, say, four black students 
in any classroom. Under these circumstances, the smallest pos- - 

sible value of t consistent with this constraint would be 

mgs 

.* Koslin, S., Koslin, B. L. and Pargament, R. Efficacy of 
Scho ol Integration Policies in Reducing Racial Polarization . 
Paper presented at the meetings of the American Psychological 
Association, Honolulu, Hawaii, September, 1972. 
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greater than itj^.^- To reflect this school policy, t 
^ mgs min j ^ xtxgs 

corresponding to the distribution in which no less than four 
black Vtudents are found in any class could be defined as 
" ^^mgs.^min* " this adjustment is made, the maximum value of 

the racial balance indicator would continue to be 100%, corres 
ponding to maximum classroom balance in the face of the policy 
constraint that no less than four black students must be in a 
classroom to avoid alienation effects.* 

To illustirate the use of the indicators > as well as the 
discussion in the previous paragraph, the classroom balance 
in the third grade of a particular school will be calculated. 
For the .purposes of this example, only one minority group 
will be "assumed to be represented (blacks :^'^=1 ) and the racial 
heterogeneity will be assumed to be 10% (i.e., the student 
population to be assigned to classrooms is 10% black) . The 
remaiiling conditions specifying the student population to be 
distributed to the third grade classrooms of schools, 
= 240 (total students in the third grade); 

N 

IS' = 24 (black students in the third grade); 

N *' 
rs = 216 (remaining, students in the third grade) ; 

C = 8 (number of classrooms in the third grade) ; 

n^ = 30. for all classrooms (class size-~the same for all 

classes in this ^-^xample) . , 

In presenting values of an indicator calculated from 

"(t ) . ", the constraint should be explicitly identified 
^ mgs min ' ^ < 

to avoid misinterpretation. 



The distributions required to \calculate the classroom 

\ • 

balance indicator in this ' example are^.shown in Figure III-4. 

Distribution 1 is the actual distribution of students to 

the third-grade classrooms of the school\ Applying the formula 

for t_ ^ to- this distribution, t = O.lll is obtained, 
mgs 138 \ 

Distribution 2 corresponds to the most probable distribution 

(-maximum classroom balance) for which the formula for t 

\ mgs 

gives (t c^min " Distribution 3 correspond&\ to the least 

i 3 s rmn \ 

probable distribution (maximum classroom imbalance) for which 

the formula for t gives (t ) v =0.379. Therefore for 

mgs 1 3 s max 

the actual distribution (distribution 1) , the classroom balance 
for black students in the third grade in this school is: 



b = 100 X 

1 3 S 



(to - t 
. 1 3 s max 1 3S 

^^iss'max ^^iss'min 



= 100 X 



0.379 - 0.111 



0.379 - 0 



= 71% 



On the other hand, if the policy of the school is to 
have no less than four blacks in each class to avoid aliena- 
tion effects, then distribution 4 , is the appropriate one from 

In this case, the classroom 
balance for black studehts in the third grade becomes 



which to calculate " (t^ 33) j^j^^^^' 



b ^ = 100 X 

1 3S 



3S^max 



IBS 



(t ) 
133 max 



"^^1 ss^min" 



= 100 X 



0.379 - 0.111 
0.379 - 0.044 



= 80% 
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Key: m=l denotes black students 

r denotes remaining students 



Fig. III~4. Examples of distributions required to calculate 
classroom balance under different conditions. 



ERLC 



- 45 - 



The value of classroom balance is higher in this "constrained" 
case because classroom balance is being determined relative 
to a distribution (distribution 4) in which some imbalance 
is being' maintained in order to avoid alienation effects, 
rather than relative to the most probable distribution 
(distribution 2) which corresponds to maximum classroom 
balance . . * " 

When only a small number of minority students are being 
distributed to schools or classrooms, large changes in the 
values of the balance indicators can arise if one or two 
minority students are shifted from one unit to another. For 
example, suppose the student population contains only one 
minority student, and suppose further that the entire stu- 
d.ent population is to be distributed to two classes of un- 
equal size'. There are only two possible distributions in 
this case because the single minority student must be in 
either one classroom or. the other. Necessarily, o.ne of the 

two possible dis tril^utions corresponds to (t <^tnd the 

^ mgs max 

other corresponds to (t • • Therefore by shifting the 

mgs min 

single minority student from one classroom to the other f 
the classroom balance shifts from 6% to 100%, or vice versa. 

Situations like this will arise when either the numti^r 
of minority students is small, or the number of available 
schools or classrooms is small, or both. In such cases the 
number of possible distributions, and consequently the number 
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of different values which the balance indicator can assume, 
is also small. . ' 

Describing racial balance in terms of balance indicators 
related to the proportions of minority students in schools 
and classrooms is simply not appropriate when only a few . 
minority students are involved. If the number "of possible 
distributions is, say, ten or less, the balance indicators 
serve little purpose. Ther assignment to schools or classrooms^ 
of the few minority students involved is best handled 
on an individual basis in such circumstances, and the use of - 
balance indicators to describe these distributions can be 
dispensed with. 



4 . Suggestions for the presentation of school and class- 
room balance information . > Care must be taken to 
describe the status of racial balance within a district 
.with precision \and without distortion. Racial balance is 
a complex and subtle matter, and poor decisions can result 
if important elements of school and classroom balance 
information are sxippressed in the decision-making process. 
The purpose of this section is to suggest ways of pre- 
senting racial balance information for decision-making 
purposes, to suggest methods of summarizing school and 
classroom information, and to discuss briefly what utility 
these suimnaries might have. The discussion is guided by the 
following precepts; 
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• School and classroom balance information. Should 
be presented separately for each minority group represented 
in the student population of the district* (see the discussion 
in Section A of this chapter) ; 

• In any presentation, the number^-of minority stu- 
dents in each school or classroom should be p2?esented along 
with the corresponding calculated value of the balance indi- 
cator. If several grades or schools in a district show a 
high degree of irnbalance, and if district resoiirces for deseg- 
regation programs are limited, the number of students affected 
by different candidate programs could be a determining factot 

~rn^decl-di-ng^wh:i-ch-~i-mbaiance--condition to address first. This 
procedure will also serve to identify grades in which the num- 
ber, of minority students is too small to permit racial balance 
indicators to be usefully employed. 

a. The presentation of school balance information . 
An example of the suggested format for the presentation' of 

.district-wide school balance information for one minority group 
is shown in Figure III-5. A similar presentation would be made 
for- each minority group attending the schools of the district. 
In this example, racial heterogeneity for the minority group, 
is- 25% for the district as a whole, but it is 45% in the 
first grade and falls to 5% in the eighth grade. It is assumed 
that the district has, eight schools', a'nd that there are eight 
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grades in each school. In practice, a digital computer would 
generate the presentation. 

The array on the left of the figure gives the num- 
ber of students from minority group m in each grade and school 
of the district. The usual marginal totals giving the num- 
bar of m-students per grade (right) and m-students per school 
(bpttom) are shown. The niimber of r-students (the "remain- 
der") per grade and per school is also shown, set off from 
the margins of the array with spaces. 

For this example/ the calculated values of the 
school balance indicator, ^j^^g' shown on right side of the 

figure along with a number called the "weighted school imbal- 
ance" (I , ) which is discussed below. The values of B show 
mb , mg 

that School balance is not particularly high in any grade in 
^ this district. The . largest imbalance exists in grades seven 
and eight where, however, the numbers of students from minoxity 
group m are relatively small. 

It is useful to construct a weighted index of 
school imbalance by grade, Ij^gf to flag extreme situations 
which may require priority action ty local administrators. 
This index will be large for grades in which school balance is 
low, or when a large fraction of m-students is affected, and will 
* _ 

I is defined as follows: 
mg 

number of studen**« from" 
minority group m in 
grade g of the district 
the number of students 
from minority group m 
in the district 
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mg 



weighted school im- 
balance by grade 



(100 - B)- 
mg 



be "doubly large" for grades in which school balance is low 
and the fraction of m-students affected is high. 

The values' of the weighted school imbalance index 
for the illustrative case discussed above are 'shown in the 
extreme right column of Figure III-5. If district decision- 
makers are interested in identifying "the largest groups 
with the greatest needs" as a basis for ordering decision 
priorities, which is what the imbalance index is designed to 
do, the imbalance index may be more useful than "^he school 
balance indicator . In the example , the third-grade imbal- 
ance index is highest, even though school balance is lower 
in other grades (the seventh and eighth) with fewer minority 
group students. 

If the district wishes to summarize the status of 
school balance for students from minority group m over all 
grades, a weighted, school balance index, E^, can be con- 
structed.* 



is defined as follows: 

n'OiTtbar of students fromY 
minority group m in 
grade g of the district 
number of biiudents from 
minority group m in the 
district 



\ 
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Fweighte'd school 1 _ X ^ . 
Imbalance, all grades! ^ 2mmti ^"^9 ^ 



all 
grades 



This index may be useful in making year-to-year or inter- 
district comparisons. It has the property that its value 
is heavily influenced by the school balance in grades con- 
taining large numbers of minority group students, and less 
so by the school balance in grades containing fewer minority 
students. The weighted school balance index for the example 
shown in^Figure III-5 is 59.9%, and appears at the bottom of 
the Figuire . 

b. The presentation of classroom balance information > 
The presentation of classroom balance information requires 
several separate displays for each minority group attending 
the schools r)f the district. A separate display of class- 
room balance information for each school in the district, 
analogous to the display of school balance information in 
Figure III-5, should be presented. A district-wide surnmary 
of classroom balance information should also be constructed. 
An example of a display of classroom balance information for 
a particular school is shown in Figure III-6. An example 
of a district-wide summary of classroom balance information 
is shown in Figure III-7. 

Figure III--6 shows classroom balance information 
for students from minority group m in school number l of 
district Y. Each grade is assumed to have 8 classrooms. ' 
Racial heterogeneity is roughly 50% in the school as a whole, 
but varies from grade to grade. Except for the omission of 
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marginal totals at the bottom of the array on the left,* the 

format is identical to the format in Figure III-5. 

The calculated values of the classroom balance 

indicator for school 1 , b , are shown on the right of the 

mg-i , 

figure, along with the weighted classroom, imbalance 'index, 

r . The values of b indicate that classroom balance 
mgi mgs 

is lowest in the fifth grade, and relatively low in the 
third and fourth grades. The weighted classroom imbalance, 
index is /highest for the fifth grade in this case. Even the 
larger number of minority students in the third and fourth 
-grades cannot offset the effect of extreme imbalance in the 
fifth grade. 



• It makes* no sense to combine students in classrooms having the same identifying number 
in different grades. The classroom number is an arbitrary identifier. 

This quantity is defined in the general case for each grade in a school (by analogy to 
the school imbalance case) as: 



mgs 



Tweighted class-1 
= I room imbalance I = 
Lby grade J 



(100 



b ) X 
mgs 



'number of students from 
minoT-ity group m in 
grade g of school s 
number of students from 
minority group m in 
school s 
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The. status of classroom balance over all grades in 
school s for students from minority group m can be summarized 

by constructing a weighted classroom balance index, b .* 

' ms 

This index may be useful in making comparisons among dif- 
ferent schools in the district (having the same number of 
grades) or in making year-to-year comparisons-. The weighted 

classroom balance index for school 1,5 = 60.6%, appears 

'mi ' t't' 

at the bottom of Figure 111-6. 

Figure III-7 illustrates a district-wide summary of 
classroom balance information. The district is assumed to 
have three schools, one of which (school 1) is the school 
used to -illustrate the presentation of classroom balance 
information in Figure III-6. The arr'ay on the left in 
Figure III-7 gives the number of students from minority 
group m in each grade and school of the district. The array 
on the right is composed of the values of the classroom 
balance indicator for each grade and school within the dis- 
trict. The data show that school 2 has perfect classroom 
balance in all grades while in school 3 classroom balance 
decreases as grade increases. Note that Grade 8 of school 3 
is completely unbalanced. 



b is defined as follows: 
ins 



fweighted class- 
1 room balance, all 
[_grades, school s 



] -E 

-* all 



all 
grades 



b X 
mgs 



number of students from' 
minority group m in 
grade g of school s 



number of students from 
minority^ group m in 
school 5 
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\ The display in Figure III-7 permits rapid associa- 

• tion of the degree of classroom balance in' any grade and 
school with the number of m-students affected. For example, 
the array on the left shows that there are 32 m-students in 
grade 8 of school 3, 

A weighted classroom imbalance index .for each grade, 
ij^g' been constructed to flag extreme situations.* Local 

administrators may wish to use these flags as a basis for 
ordering decision priorities. A weighted classroom imbalance 
index for each school, i^^ , has also* been constructed. This 
index could serve to identify schools which might require 

i 

priority action by local administrators. 



is defined as follows: 



mg 



[weighted class- 
room imbalance 
index by grade 



1 



all 
schools 



' (TOO - b ) 
mgs 



number of students from' 
minority group m in 
grad^ g of school 



number of students from 
minority group m in 
grade g 



^ms is Refined as follows: 



[weighted class-"] 
room imbalance 
index by schoolj 



all 
grades 



(TOO 



mgs' 



'number of students from' 
minority group m in 
grade g of school s 



number of students from 
minority group rr\ in 
school s 



i 

1. 
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These imbalance indices have been calculated for the 
data shown in Figure III-7. Their values for each grade and 
school are entered at 'the right and bottom margins of the 
array of classroom balance indicators in the Figure. The 
classroom imbalance index by grade permits the fifth grade ^. 
to be identified quickly as the grade which probably should 
receive high priority if desegregation programs by grade 
are being planned by the district. The classroom imbalance 
index by school permits school number 1 to be identified 
quickly as a possible "^firsfe target if desegregation programs 
by school are being planned by the district. 

For purposes of making the year-to-year or inter- 
district comparisons described earlier, a weighted class- 
room balance indicator, b , which summarizes the status of 

m 

classroom balance for students, from minority group m in all 

grades and schools of the district, can be constructed.* 

The value of this index for the example given in Figure 

III-'V, b = 71.4%, is entered at the bottom of the Figure, 
m 



b is defined as follows: 
m 



["weighted class- 
S = room balance, all 



Lgrades and schools^ 



all all 
grades schools 



mgs 



number of students from' 
minority group m in 
grade g of school s 



number of students from 
minority group m in the 
district 
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C. The Role of Racial Balance in the Monitoring and 
Evaluation of Desegregation Progra ms. 

Racial balance indices fill two important roles in the 
monitoring and e;valuation of desegregation programs. First, 
they are themselves the effectiveness measures rWhich determine 
the degree to which a desegregation program has been success- 
ful in accomplishing its objectives for structural change in 
the schools and classrooms of a district; or, if a^ desegrega- 
tion program extends over several years, they measure the 
progress which is being made toward the attainment of the 
desired degree of structural change. Second, racial balance 
indices, together with racial heterogeneity indices, consti- 
tute the minimxim set of environmental variables required to 
determine the cognitive and ncncognitive effects of a dese- 
gregation program in the second phase of evaluation (see 
Chapter II) . 

The data required to determine school and classroom 
balance indices (as well as racial heterogeneity indices) 
have been specified earlier in this chapter. It is only 
necessary to know the number of students from ^11 groups 
(minority and majority) in each grade in each school, and 
in each classrooi^of each grade within each school, to 
calculate these ind^ices, . However, the crux of desegregation 
li'es in the cognitive, and ncncognitive changes produced in, 
the behavior of participating students, and sufficient 
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information should be gathesred by districts to permit not 
only the calculation of racial balance indices but also the 
evaluation of these behavioral . changes . 

In Chapter V, the major types of evaluative study de- 
signs are described in detail. Each requires the collection 
of different elements of information, but all require more 
information than .is necessary just to calculate racial balance 
indices. In particular, as indicated in Chapter II, informa- 
tion on^ student characteristics is required, and baseline 

data characterizing predesegregation environments is '^ssen- 

1 

tial. • ' 

Not all districts may have the administrative and 
technical resources to undertake the collection of data re- 
quired for the evaluation of desegregation programs. Further- 
more the calculation of racial balance indices will require 
computer programming and data processing capabilities which 
are not available to many districts. 

In order to facilitate valid desegregation program eval- 
uations, the State Education Department is prepared to pro- 
vide assistance to districts in the - collection of pre- and 
post desegregation data required for evaluation studies, and 
in the calculation) presentation and interpretation of racial 
balance indice^.. The State Education Department will gener- 
ate forms for the collection of the environmental and per- 
sonal data required for . evaluation studies, and will assist 

■ . ■ - 
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the districts in the implementation of the data collection 
process. 

Using the collected j3ata, the Department will generate 
the presentations of school and classroom balance information 
that are de'scribed in Section IIIB. These presentations will 
be provided to the districts, accompanied by analyses which 
may be helpful in the interpretation of the racial balance 
information contained in the presentations. 

A comprehensive list of the personal and environmental 
data required to undertake both stages of desegregation 
program evaluation, and to facilitate interpretations of the* 
results / is given below. Some elements of information are 
already collected by the Basic Educational Data System (BEDS) 
and are included in the list for completeness. 

1 . Information to be collected from each school . 

a. School code School code may be taken from the 
district's BEDS report. o 

b. Ethnic or racial composition of professional 
s taff * This is the same as' Item #3 on the BEDS school data 
form. 

c . Ethnic and racial composition of the student 
body » Number (per grade) of white, black, Puerto Rican, and 
other pupils. This is the same as Item #4 on the BEDS school 
data form. 



d. ^^ The proportions of boys and girls (by grade) 
in the student body , 

e. Records of student IQ and prior academic 
achievement (if measures or indices are comparable for all 
students) 

f. Teacher assignments . Identifies, grades where 
teachers are assigned to the same class all day (^elf-con- 
tained classrooms)" as opposed to grades where teachers or' 
children or both change classes during the, day. 

g. Socioeconomic data . Estimates of the ^percent 
of students from low, middle, and high SES (socioeconomic 
status )■ homes c 

2 . Information to be collected from^ each classroom 
-tea^cher . . ' ^ 

a. School code . The school code would he pre- 
entered on a form which the teacher fills out. 

b. Teacher name > The, teacher name can also be 
pre-entered on the form. " 

c . ^ Type of as s i qnme nt . Does the teacher spend 
the entire day with one class or does she or he have several 
teaching assignments? In the latter case, does she or he 
teach a special siibject (e.g., art, physical education) 
where students are assigned to self --contained' classrooms 
with one teacher all day, or does every teacher in that 
grade or school have multiple assignments? This information 



- 6 2 - 



is needed to allow instructional units to be i'dentified and 
to make sure that such units are not duplicated in ^^alance 
computations . *' \ . ■ 

d. For each class taught : 

1 . Grade level 

2. Class enrollment list and total number of 
students . , • ' ' ' 

3. Sex and ethnicity (or race) of each student 
Total number of white, black, Puerto Rican, Indian and other 
pupils • 

4. Number of hours per week that the teacher 
meets with the class. y 



Chapter IV 

Cognitive and Noncognitive Effectiveness 
Measures for Desegregation Programs 
Once it has been determined that implementation of a 
desegregation program has resulted in the desired change (s) 
in the racial composition of educational units, /the. district 
may proceed to evaluate the effects of the desegregation 
program on students* cognitive and noncognitive development. 
Chapter IV is designed to provide districts with guidance in 
selecting effectiveness measures in both the cognitive and. 
noncognitive domains. 

,/' 

In the cognitive area, where there are many tests and a 
long tradition of educational measurement, the^^ discussion will 
focus on two major ^types of cognitive tests (norm referenced 



some limi 



and criterion referenced) , with emphasis on some limitations 

/ 

of each type in the evaluation of desegregation programs . 

In the noncognitive area, where fewer tests exist and where ^ 

-the tradition of measurement (at least ir^/ the context of 
program evaluation) is weaker, the rationale for including 
noncognitive measures in the ''evaluation of desegregation 
programs will be reviewed, criteria for noncognitive effec- 
tiveness measures will be outlined, and two racial attitude 

.tests which meet these criteria will be described. 
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A, C ognitive Ef fedtiveness Measures' 

In evaluating desegregation programs, cognitive tests 
are used to determine the effects of the programs on students 
achievement in one or more areas of the academic curriculum, 
e . g -reading , mathematics, science, etc,' Achievement in 

these a^reas is clearly ' important to those who manage the 

" - ■• - -J 

schools. 

1. Scope of this discussion . An extensive literature 

exists on the theory and practice of achievement testing. 

k ■ ' 

Any attempt to review or even to summarize that literature 

* 

far exceeds the scope of this chapter. 

Specific standardized achievement tests, such as the 
Stanford Achievement Tests, the Iowa Tests of Achievement, 
etc. will not be discussed here either. These tests are 
already well known to school districts. Moreover, a hand-- 
book designed to assist school districts in selecting &- 
standardized tests is now available. This handbook supple- 
ments, in a practical way, information provided by test 
publishers and technical evaluations of tests available in 

Interested readers are referred to Cronbach, L.J. 
Essentials of psychological testing . (3rd ed.) New York: 
Harper and Row, 1970, and to Thorndike, R.L. (Ed.), Edu - 
cational measurement , (2nd ed.) Washington: American 
Council on Education, 1971. 

if "k 

Hoepfner, R. (Ed.) , CSE elementary school test evaluations 
Los Angeles: Center for the Study of Evaluation OUCLA) , 1970 
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reference works. / 

/ 

This section of Chapter IV focuses on some limi-/ 
tations of two^'major types of cognitive tests (norm, referenced 
and criterion referenced) as effectiveness measures in the 
evaluation of desegregation programs, and notes the conse- 
quences of these limitations on the district * s selection of 
a design for the evaluation study. 

2. Norm referenced tests . The standardized achievement 
tests familiar to all school districts are norm referenced. 
A student's test performance is interpreted by comparing it 
with the performance of other students, usually with- -the^"^" 
performance of the students on whom the test was originally 
nor me d . 

A student's raw score on a standardized test (i.e-,, 
the number of items answered correctly ). has no absolute 
meaning; it does not directly tell what he knows. A student's 
raw score acquires meaning, only when it is compared with the 
raw scores of other students. This compari^son is usually 
expressed as a grade equivalent score (grade norm). A grade 

equivalent score (grade norm) simply identifies the grade 

/' 

level at which a raw score was the mean score of students 
in the norming sample. 



Euros, O. K. (Ed.) The mental measurements yearbooks. 
Highland Park, N. J.: Gryphon Press, 1941-1965* (irregular). 
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In evaluating the effects of desegregation on achieve- 
ment,' norm refer^ced tests assess whether or not desegregation 
causes a change in the achievement of desegregated students 
relative to the achievement of other students. The ability 
to detect such a change will depend both on the test selected 
:and on the group (s) used for comparative purposes. 

a. Test selection . All major standardized achieve- 
ment tests are sufficiently reliable for group comparisons of 
the type carried out in evaluating desegregation programs; 
hence, reliability is not an issue in test selection. How- 
ever, the validity of the test (whether the test measures what 
the district wants. to measure) is ^a matter of serious concern. 

The district must try to select a test whose 
item content matches its own program reasonably well. If 
the test does not measure what the schools are teaching, a 
change in the number of items answered correctly (and hence 
a change in relative achievement vis-a-vis other students) is 
unlikely. Even if students are, in fact, learning more after 
being desegregated, the poorer the match between test content ^ 
and the district's program, the less sensitive the test is 
..likely to be in detecting achievement changes which result 
from desegregation. 

/ ' ■ ■ 

Item content should also be examined because test 

names often do not provide an accurate description of what the 
tests really measure. Striking examples of test items meas- 
uring skills other than those indicated by the test title 

- 67 - 



are found in some reading tests, where items that are supposed 
to assess reading comprehension in fact measure abstract 
reasoning skills. The effects of a desegregation program on 
achievement in a curriculum area cannot be accurately assessed 
if items on the test are measuring something else. 

In addition to examining item content before a 
test is selected, a district should examine the test's format 
and instructions. The effects of a desegregation program on 
achievement in a subject matter area may be masked if, the 
test format or directions are too complex. Even if deseg- 
regation does affect h6w much students have learned in the 
area, if students cannot understand test instructions or are 
confused by test format, they will be unable to demonstrate 
their higher achievement on the test. In short, an overly 
complex test format or overly difficult test instructions 
will probably make the test insufficiently sensitive to 
detect changes in achievement in the content area. This 
problem is especially serious in the primary grades. 



Since the function of norm referenced tests is to compare 
the performance of students, these tests are constructed for 
maximum accuracy in making distinctions between people. 
Therefore, when a norm referenced test is constructed, final 
selection of test items is determined largely by statistical 
properties of the items. The mismatch this may cause between 
test title and item content is illustrated in Klein,; S. P., 
The uses and limitations of standardized tests in meeting the 
demand for accountability. UCLA Evaluation Comment, 1971, 2 
(no. 4) , 1-7. 
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b. Comparison groups . Assuming that the district 
succeeds in selecting a test which is valid in terms of test 
content and format, the ability to. detect and accurately 
assess changes in achievement associated with desegregation 
will depend, on the availability and appropriateness of scores ' 
with which to compare, desegregated students' scores. 

The ability to detect a change in achievement 
requires, first, that there be base--line data for comparative 
purposes and, second, that the base-line data belong to a group 
with^ which it is reasonable to compare desegregated students' 
scores . ' 

Without .base-line data there are no grounds for- 
concluding that desegregation has or has not had any effect 

on scores, since there is no way of knowing what students' 

/ 

scores would have been without desegregation. ' Comparison' 
with national norms (i.e,, treating national norms as base- 
line data) is appropriate only if district students closely 
resemble the norming sample and typically have test scores ^ 
similar to ^those of the norming sample. To the extent that 
the district's students differ from the norming sample in 
such characteristics as IQ, SES/ curriculum, length of time ; 
desegregated, etc., they may be expected to depart from 
national norms. 

Accurate interpretation of changes in district 
performance under desegregation therefore requires base-lirie 



data from the district itself, either in the form of local 
norms or control groups. Local norms, developed over a period 
of years, provide data concerning what scores of students in 
the district were like prior to the desegregation program, 
and thus make it possible to detect a change in scores fol- 
lowing desegregation. If local control groups are used, base- 
line data are obtained from groups of students similar to 
those desegregated but not involved in the desegregation 
program. With either type of base-line data, comparisons are 
made between comparable groups, such as between segregated / 
black. id desegregated blacks, or between segregated whites 
and desegregated whites. 

3. Criterion referenced tests . In recent years (largely 
as a consequence of increased need to evaluate the outcomes 
of individualized ^ ins truction). attention has been drawn to a 
type of test in which a student's performance is directly 
interpreted with respect to how well he has mastered the 
subject matter being tested, rather than with respect to how 
his test score compares with the scores of other students. 
Tests of this type are called "criterion referenced," since 
they are built to provide a direct measure of the extent to 
which students have acquired the cr-iterion skills and knowl-- 
edge specified by curriculum objectives. Because items on 
a criterion referenced test are directly related to mastery 
of specific subjep;t matter, a score on a criterion referenced 



test is directly interpretable in terms of what students have 
learned. 

Tests of this type have been routinely used by 
teachers for many years • A good example is the weekly spelling 
test used to determine how well students have learned to spell 
the words studied during a particular week. Students' scores 
on such tests are interpreted not with respect to how well 
other students did on the same test in other schools or in 
previous years, but simpl^^ in terms of how much students ' learned 
of what the teacher tried to teach during the period of in- 
struction. 7 

In evaluating desegregation programs, criterion ref- 
erenced tests are used to assess the effects of desegregation 
on mastery of the curriculum taugkt in local schools.. Com- 
parison groups are required for this purpose. Even though a 
single criterion referenced score has ' an interpretable meaning, 
knowledge of desegregated students ' mastery level does not 
constitute adequate data for evaluation purposes. 

In order to demonstrate that desegregation affects 
test scores, a comparison must be made between the observed 
(actual) scores of desegregated students and their prior 
scores or the scores they would be expected to have if they 
were not desegregated. The district must determine how the 
number (or proportion) of desegregated students achieving 
various levels of mastery on the criterion (e.g., 100% correct. 
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90% correct, 80% correct, etc.) compares with the number (or 
proportion) of segregated students achieving those levels. 

To derive interpretable data from a criterion ref- 
erenced test in evaluating a desegregation program, the test 
must be held constant across the groups of students who are 
compared. Thie requirement for test constancy imposes the 
further requirement that curriculum content be the same across 
the . groups . 

If grade level can be held constant in the evaluation 
study, it is fairly simple to use criterion referenced tests 
as effectiveness measures . Under these conditions, if the 
whole school district undergoes desegregation, scores of 
students in a particular grade prior to desegregation can be 
compared with scores of students in that same grade following 
! desegregation (e.g., third graders prior to desegregation are 
compared with third graders following desegregation) . Alter- 
natively, if only part of the district undergoes desegregation. 



Comparing numbers (or proportions) of students attaining 
various levels of mastery is preferable to comparing untrans- 
formed mean scores because means are"" constrained by the 
"ceiling" of the test, i.e., there is no opportunity for stu- 
dents attaining complete mastery to improve their test scores « 

It is not necessary that exactly the' same test be adminis- 
tered each time. Alternate forms of the test, constituted by 
randomly selecting a fixed number of items per objective, are 
permissible. They may be used to avoid or solve problems of 
test security, i.e., to remedy situations in which knowledge 
of test content leads teachers to "teach for the test." 
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scores of students involved in the program can be compared with 
scores of those who are not involved (e.g., third grade program 

r 

participants versus third grade nonparticipants) . In both 
instances, scores of segregated students are used to provide 
base-line data for purposes of comparison. 

If grade level is not held constant in the evaluation 
study^ criterion referenced tests may still be used as effec- 
tiveness measures-, but base-line data for all grade levels in- 
volved in the study must be collected prior to desegregation. 
For example, a district may ask: "how does the achievement 
of third grade students prior to desegregation compare with 
the achievement of the same students as fourth graders one 
year latjr, following desegregation?" 

To answer this question, the district must adminis- 
ter the criterion test to students in fourth grade prior to 
implementation of the desegregation program, so that base-line 
data aire obtained for purposes of later comparison. 

Moreover, if the district wants to determine whether 
desegregation has a cumulative effect on achievement (i.e., 
whether achievement gets progressively better for students the 
longer they are desegregated) , base-line data must be collected 
prior to desegregation in all grades where postdesegregation 
testing will eventually take place . 

,?4'.. Cdnclusion . Though norm referenced and criterion 
referenced tests differ in purpose, construction, and 
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interpretation, they are alike in that both require the use 
of local norms or control groups to assess the effects of 
desegregation on students' academic achievement • 
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B. Noricognitive Effectiveness Measures 

1 . Importance of assessing noncognitive program outcomes . 
In the evaluation cf desegregation programs , noncognitive^ tests ' 
are used to assess the effects of the program on students/ 
attitudes. Since the development of more favorable inter- 
racial attitudes among students is frequently a' goal of deseg-- 
regation programs, the tests are most often used to determine 
how interracial attitudes have been affected by the desegre^ 
gation program^. While desegregation may im^^rove interracial 
attitudes, neither* the empirical evidence /nor social psycho- 
logical theory suggest that all desegr^^'gation programs will 
necessarily result in such improvement./ Therefore, each school 
district must assess the effects, of ii/s own desegregation pro- 

c 

gram on the racial attitudes of its students. 

Attitudes should also be assessed because they may 

play an important role in mediating (i.e., influencing) achieve- 

*** . 

ment. In one major educational survey - it was found thatia 



* Carithers, M.W. School desegredation and racial cleav^-age, 
1954-1970: a review of the literature. The Journal of Social 
Issues , 1970, 26^, 25-48. 

** • . ' 

Pettigrew, 'T,F. Race and equal educational opportunity. 

Harvard Educational Review , 1968, 38 , 66-76. 

ic i€ ic ^ 

Coleman, J.S., et al. Equality of educational opportunity ♦ 
Washington: U.S. Government Printing Office, 1966. 



ERIC 



- 75 - 



combination of several attitudinal factors was better able - 
to predict students' verbal achievement scpres than were 
all school factors (e.g./ school facilities, teacher charac- 
teristics, per pupil expenditures^ otc . ) combined. The 
survey also suggested that interracial attitudes may be a key 
variable in determining the effects of desegregated schodl ^ 
environments on black achievement. It was found that those 
black students reporting the greatest proportion of close 

white friends also had the highest achievement scores . Further 

* ' ' .. . ^^^^^ 

more, black achievement was best in schools reporting the^^" 
smallest amount of interracial conflict. Thus, in e^^afTua- 
ting desegregation programs, assessment of racia'l attitude 
changes may facilitate understanding of changes in academic-^ \ 
achievement.' 

Districts may also wish to use noricognitive measure^ 
to assess prograra effects on other attitudes, such as self- 
concept, achievement motivation, and sense of fate control, i 
which are important in personal-social development and which 
may affect achievement. 

2 . Criteria for noncognitive effectiveness measures . 
To function as an - effectiveness measure in the public schools, 
a test should be group administrable in the elementary grades, 
should require low levels of .reading skill and verbal compre- 
hension, should be relatively resistant to socially ' compliant 



responses, should be objectively scorable, should be reliable, ^ 
and should have demonstrated validity for purposes of program 
evaluation. ^- ^ 

Very few noncognitive tests have been developed which 
meet all the above criteria. The discussion in this section 
of Chapter IV focuses on tij^ People Test and the Pick-A-Class 
Test, two currently available measures of interracial atti- 
tudes which do meet the criteria. These tests may/be obtained 
through the State Education Department, which will (at the-/ 
district's request) aid in scoring the tests, carry out com- ^. 
puter analyses of the data, and interpret 4:est results for- ^ 
school districts. -"'^ 

3. The People Test; A measure of social distance . The 
People Test is a nonverbal measure of students' social 
distance attitudes and beliefs. Social d|.stance refers to ■ 
the degree of intimacy or closeness in interpersonal in- 
teractions which members of one group are willing to share 
with members of another group. Social distance stems from ^ , 
the distinctions which a culture makes between people with 
respect to .such dimensions (characteristics) as race, sex, 
age ^ ethnicity, etc., and is believed to jbe a universal 
phenomenon . ^ 

The People Test is specifically concerned with the 
social distances which students attribute to the socially 
important dimensions of race and sex. The principal use 

J- 

- 11 - 



of the People Test in evaluating a desegregation program 
is to determine whether and to what extent the program 
results in a change in these distances. Because patterns 
of racial prejudice in the ynited States traditionally have 
had itnportant sexual components, the People Test also assesses 
- sexual social distances and racial-sexual social, distances • 
a. Method ('and format . ^ Social^ distance concepts are 
assessed by asking that students judge the relative degree of 

similarity between figures (drawings) of childrei^ who differ 

t' ■ 

^ by race, by sex, or by both race and sex. Students are in^ 
structedito express the degree of similarity between any two 
figures by ineans of the amount of space they place between 
those figures. It is assumed that in making, these judgments 
the child transforms the distinctions he has learned to draw 
between people into metric distances , and that ' the smaller 
the distances placed between any two figures, the smaller 
*the social distance between the people represented by those 
figures. The distances employed in the test are not assumed 
to 'have any absolute meaning in teifins of feet, yards, etc.; 
rather,- it is assumed that their relative size reflects tH^e 

^ comparative importance of race, sex, and race-sex as characr 
teristics by w)Mch distinctions are made between people. 

The figures used in the test are line drawings of 
a white boy, a white girl, a black boy, a black girl, and a 



\ 

\ 



stick figure representing the self. These figures are age- 
appropriate for students taking the test (there are three 
versions of the test, one each for grades 1-4, 5-8, 9-12). 

Students are instructed that the greater the 
degree of similarity between any two figures, the closer 
together the figures should be placed; the smaller the de- 
gree of similarity between the figures, the further apart 
they should be placed. One figure is printed on a pressure 

. sensitive tab, enabling students to paste it at whatever 
distance they wish from the other figure, along the response 
scale provided. 

b. Two types of test items . In' some test items, 
students judge the similarity between two figures (excluding 
the self figure) which differ from each other by race, by 
sex, or by race and sex. Responses to these test items 
reflect students' concepts concerning the racial and sexual 
distinctions which are ordinarily made between people in our 
culture. These items thus assess baii^^fs concerning the 

-^cially standardized or normative distances between the 
races and the sexes. * ' 

, In other test items, students judge the simi- 
larity between themselves (self figure) and figures of chil- 
dren who differ from themselves by race , by sex^ or by race 

and sex. Responses to these test items reflect students' 
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concepts of the relative importance of race and sex differences 
between themselves and each of the other figures. These items 
thus assess the degree to which personal attitudes concerning 
distance from others vary as a function of the race and/or 
sex of the other person. 

c- Administration . The People Test is group adminis- 
tered and requires approximately 15 minutes for students to 
complete. Students need not be able to read at all in order 
to take the test. 

The test may be administered by classroom teachers 
or by other school personnel. Test administration involves no 
special skills , other than a careful reading of the adminis- 
trator's manual prior to the testing session. About 15 minutes 
are required for the admini'^ trator to familiarize himself 
(herself) with the manual. / 

During the testing session the administrator uses 
the manual to guide students through the test. He (she) identi- 
fies the figures, explains and illustrates how distance along 

/' 

the response scale is used to repre;5'ent the degree of simi- 
larity between figures, and helps students to complete one or 
two practice items. Students then complete the test on their 
own . 

d. Scot Ing . Scoring of t:he People Test is completely 
objective, and consists of recording the amount of distance 
placed between the two figures in each test item. Since a 
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metric "ruler" is printed beneath the test items, the task- of 
recording the distances between figures is a simple one: the 
scorer need only look below the figure on the pasted tab to 
find the score for that item. Scores for each: item may 
either be recorded on a roster sheet for later keypunching or 
may be keypunched directly from the test booklet. 

\ e. Analysis of test data . People Test data may be 

analyzed to determine whether (and to what extent) the deseg- 
regation program affects the distances which students place 
between the races anc^ sexes (social distance beliefs) as well 
as to assess the program's effects oh^-personal distances from 
peers who /iiffer by race, by sex, or by race and sex (social 
distance itudes) . Data may also be analyzed to determine 
whether desegregation alters the extent tc5~~which 'a student's 
social distance beliefs. and attitudes correspond. 

Like scores on norm referenced cognitive tests 
(discussed earlier in this chapter) , People Test scores have 
no absolute meaning; they are not directly interpretable in 
terms of^what students think or feel. Intexp-retation of stu- 
dents' People Test scores requires a comparison with the 
scores of other students. Therefore, local norms (test scores 
prior to desegregation) or control group data (scores of 
students who have not been desegregated) are needed to deter^ 
mine program effects on People Test scores. 

/ 
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A total score. is not calculated for the People 
Test. Instead, analyses are based on students' responses to 
individual test items. Before substantive analyses are 



One way of structuring these analyses is by examining the 
responses of all students to any given test item. For those 
test items which do not include the self figure (i.e., for 
the normative distance judgments, such as the distance be~ 
•.tween a white girl fiaur^ and a black airl fiaure, or be- 
tween a white boy figure and a black boy figure, etc.) this 
type of analysis yields interpretable data, since the item 
has a reasonably constant meaning for all students. 

However, for those test items involving the self figure 
(i.e., for personal distance judgments, such as distance of 
self from black boy, distance of self from white girl, etc.), 
the meaning of any item varies according to the race and sex 
of the student who is taking the test; thus interpretation - 
of responses to any given test iteni is difficult. Accordingly, 
a procedure has been developed for analyzing personal distance 
judgments^ so that the results will have ^ common meaning for 
all studeitts. 

In this procedure, the test items, used in analyses vary 
according to the race and sex of the student taking the test 
and according to the type of personal distance being studied. 
For example, to study personal. ddTstance from opposite race 
peers of the same sex, the pertinent data for students who 
are black boys would be distance scores on the self-white 
boy item; the pertinent data for students who are black girls 
would be distance scores on the self-white girl item*, and so 
forth. Following the same logic, it is also possible to 
analyze other personal distances (e.g., distances from peers 
of opposite sex but same race, and distances from peers of 
opposite race and opposite sex) . Analy^s of variance- is the 
statistical procedure most' often used in carrying out these 
analyses . 

It is . also possible (by covariance analyses ) to statis- 
tically remove from personal distance judgments the influence 
of normative distance judgments. In addition to providing 
a purer measure of attitude than is obtained from a simple 
analysis of personal distances, covariance procedures also 
effectively control possible student tendencies to restrict 
judgments to particular segments of the response scale. 



- 82 - 



begun, a coinputer program is used to identify and delete any 



students whose pattern of responses suggests that they have 
not followed test directions • 

f. Validity. Because the People Test " is presented 
as a cognitive task (i.e., students are asked to judge how 
alike or similar two people are) , the actual purpose of the 
test as a measure of - attitude is somewhat disguised. To. the 
extent that the disguise is successful, the test will not in- 
duce student anxiety about revealing attitudes and the likeli- 
hood that students will give socially desirable (rather than 
"true") responses will be reduced. 



While students are free to use any distance they choose for 
each test item, it is assumed that if they understand and fol- 
low directions the magnitude of their judgments will vary 
across the items. Therefore, all test data of students showing 
little or no variability in responses across iten.s are deleted 
from analyses . 

However, even if some students do recognize the test as an 
attitude measure, it is more difficult to deliberately distort 
responses on the People Test .than on most other attitude scales. 
Students who try to disguise their attitudes by the simple de-* 
vice of pasting the tabs at the same distance for all test items 
are easily detected and deleted when the data are initially 
screened. If students attempt to distort their responses while 
varying their distance judgments from item to item, the fact 
that the figures vary simultaneously in two dimensions makes it 
very hard to intentionally distort responses in one dimension 
(e.g. , race) without simultaneously distorting responses in the 
other dimension (e.g. , sex)r. The fact that both the race and 
sex dimensions are consistently Recovered when People Test data 
are analyzed (see below) indicates that most students do respond 
honestly to the task. While it is . theoretically possible for 
students to alter responses in one dimension without impairing 
the integrity 'of judgments fcj the other dimension (thereby 
"faking" the test without being detected) , few students could be 
expected to formulate and follow the complex rules which. would 
be required to do so. 
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Several studies have provided evidence concerning 

. the construct validity of the People Test. It has consist- 

ently been found that students systematically utilize the 

race and sex dimensions in the stimulus figures when making 

★ * 

their judgments of personal and normative social distance. 

*** 

One important construct validity study in- 
vestigated developmental changes in People Test normative • 
distance scores for a sample of over 4,000 black and white 
students in grades 1-12. The developmental data were analyzed 
to determine whether they were in accord with known sociali- 
zation patterns in the United States. It was found that 



See, for example r .Koslin, S., Koslm, B, L., Pargament, R. 
and Waxman, H. Classroom' racial balance and students' inter- 
racial attitudes.. Sociology of Education ^ 1972, in press. 

Among the normative distance judgments, those test items 
which involve figures differing in two dimensions always re- 
ceive larger distances than those test items where the figures 
differ only in one dimension. In judgments involving the self, 
distances are clearly related to the degree of similarity be- 
tween the characteristics of the test-taker '^nd the character- 
' istics of the target figure. Boys consi\stently place the self 
figure closer to male stinliuli, whereas gkrls place the self 
figure closer to f emale .^imuli . Black students place the self, 
-closer to black than to white target figures, whereas white stu- 
dents ^do the^opposite. When multidimensional scaling procedures 
have been used to analyze the data, it has been found that stu- 
dents appropriately locate the self figure in the "space" cre- 
ated by' the shared properties of the other stimuli. 

* ■ — — \ 
Koslin, S., Koslin, B. L., Pargament, R. and Bird, H. Chil- 

■ dren's social distance construct^! : a developmental study. Paper 
presented at the meetings of the American Psychological Associa- 
tion, Washington, D.C., September 19 71. 
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social distance between the races" increases with age and is 
greater between opposite race girls than between opposite race 
boys ♦ Moreover, it was found that social distance between the 
sexes decreases following a preadolescent rise, and is largisr 
between whites than between blacks. In general, it w^s found 
that black students attribute greater social distance to ra- 
cial" differences than white students do. All these findings 
(as well as others too complex to review here) correspond 
sufficiently well to known socialization patterns in our cul- 
ture to provide strong support for the construct validity of 
the People Test. 

* Another validity study showed that People Test 
scores are related to scores on other racial attitude tests. 
Personal distances were , compared for students who had shown 
extreme pro-white or pro-black preferences for teachers and 
classmates on a test of racial preferences. It was found 
that students with pro-white preferences placed significantly 
less distance between the self and the white girl figure than 
4id stude^its with pro-black preferences; students with strong 
prbrblack preferences placed significantly less distance 
between the self and the black girl figure than did students . 
with pro-white preferences. Corresponding trends appeared 
for male stimulus figures. Thus it was conclude^ that 

* Koslin, S. The measurement of schoolchildren ' s racial atti- 
tudes: a validity study. Paper presented at the meetings of 
v.he Eastern Psychological Association, Philadelphia, 1969. 



students' personal distances from peers of the opposite race 

are significantly related to their preferences concerning 

classroom racial composition. 

The People Test has beeri^ used in several studies 

designed to in-^'estigate .the effects of student assignment 

policies on interracial attitudes. In one study it was found 

that normative and personal distances are smaller in schools 

with balanced than in those with unbalanced classes, espe- 

cially when the figures being judged differ by both r. ze and 

sex. In a second study it was found that personal racial 

distances are smaller in racially mixed (heterogeneous) schools 

than in racially isolated (homogeneous) schools within the 

* * * 

same district. A third study found ,that when the pro-\ 
portion of blacks in a class drops below a certain level, the 
personal distance of black students from opposite race peers . 
increases . 



Koslin, S., Koslin, B. L., Pargament, R. and Waxman, H. 
Classroom racial balance and students' interracial attit^.des. 
Sociology of Education , 1972, in press. 

** Koslin, S., Amarel, M. & Ames, N. A distance measure of. 
racial attitudes in primary grade children: an exploratory 
study. Psychology in the Schools , 1969, £ 382-385. 



Koslin, S., Koslir, B. i.. , and Pargament, R. Relationships 
ween educational integration policies and. students ' racial 
altitudes. Paper presented at the meetings of the American 
Educational Research Association, New York City, February, 19.1 
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Thus, in policy-related studies to date, the 
People Test has proved sensitive to differences in classroom 
racial balance, school racial heterogeneity, and classroom 
racial heterogeneity. 

g. Reliability . Test-retest studies have shov/n 
that the People Test is adequately reliable for analyses of 
group data. Reliability data are based on test-retest studies 
in which alternate forms of the People -Test were administered 
to approximately 600 students in grades 3, 5, and 9, with a 
one-month time interval between testings. While it was found 
that item reliabil. "-.ies vary somewhat according to the typie . 
of item and age of subjects, they are high enough for group 
comparisons of the type carried out in the evaluation of 
desegregation programs . 

The Pick-A-crass Test . The Pick-A-Class Test is a 
semi --disguised measure which, assesses children's racial pref- 
erences for classmates and teachers . In evaluating the 
effectiveness of a desegregation program, the Pick-A-Class 
Test is used to measure differences between the classmate 



Items measuring personal social distances are more reliable 
than those measuring normative social distances (average item 
reliability is r = .63 for personal judgments and r = .42 for 
normative judgments). Item reliabilities are higher for fifth 
and ninth graders than for third graders (average .item relia- 
bilities in those grades are, respectively, r = .56, = .56, 
and r = .39) • Since these reliability coefficients are based 
on individual test items, father than on total test score, they 
can be considered acceptably high for group comparison purposes. 
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and teacher preferences of desegregated students dnd those of 
segregated students,- or to assess differences between students' 
preferences prior to and following desegregation. The test is 
designed for use in grades 1-4, 

a. Format and method . The test materials are 18 
sketches of classroom scenes. The sketches vary systematically 
in the type of activity portrayed (three activities are shown) , 
in the race of teacher portrayed (two variations: black and 
white), and in the racial composition ot pupils portrayed 
(threR variations: all white^ qill blacky racially mixed) . - 
The rs sketches are organized into nine pairs so 
that all possible comparisons are made between the different 
racial compositions, with classroom activity counterbalanced. 
On any page, two of the six facial compositions appear, each 
embedded in a different classroom activity. In some com- ^ 
parisons teacher race is held constant while pupil' race and 
classroom activity vary. In other comparisons pupil race is 
held constant v^hile teacher race and classroom activity vary. 

: On each page of the test booklet, students indi- 
cate with a check mark the class that they would prefer to 
be in. After they have indicated their preference on all 
pages, students go through the entire, test booklet again and 
on each page indicate by means of an "X" the class which they 
would prefer not to be in. 
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b, • Adminis tration ^- The Pick-A-Class Test is group - 
administered and requires approximately 10 minutes for stu~: 
dents to complete. Students need not be able to read at all 
'to take the test. ' 

The test may be administered by classroom teachers 
or by other school personnel. Tes'jt administration reguires' no 
special skills, other than a caref|il reading of the brief 
administrator's manual prior to thel, testing session. About 
ten minutes are required for the administrator to familiarize 
himself (herself) with the manual. \ ' 

During the testing session the administrator uses, 
the manual to illustrate at the chalkboard what students need 
to do, and to guide them through the tCirst two pages of the 
test.- Students then complete the resl of the pages in the 
te;st booklet on their own. The entire! procedure is repeated 
^ when students go through the test the second time (to mark on 
each page the class that they would prelfer not to be in) . ;^ 
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c. Scoring . The test is scored by first recording 

\ 

which sketch was selected as preferred and as not preferred on 
each page. This information may be rost'ered for latea? key-., 
punching or may be keypunched directly from the test booklet. 
Students who have selected the same sketch as preferred and as 

not prefex-red are detected and deleted at this point. ^ 

■ ■ . . • • i ■ ■■ 

On the basis of his pattern ^pf responses , each 

student is assigned a score for teacher preference, a 3core 

for peer preference when a white teacher is portrayed, and a 

- ■ - 89-- ' 



Score for peer preference when a black teacher is portrayed.* 

d. Data analyses . Like scores on criterion 
referenced tests .{discussed earlier 'in this chapter), scores- 

_pn the Pick~A-Class Test are directly interpretable. However, 
to determine whether desegregation affects test— scores , com- 
parisons must be made between .the scox'es of desegregated* 
students and their prior scores or the scores they would be 
expected to have if they were not desegregated. Local norms 
p.r control groups are required for such comparisons. 

. {Separate analyses are carried put f or s tutien-^s ' 
preferences concerning teacher race ,^ classmate race when a 

white teacher is portrayed, .and classmate race when a black . 

** 

teacher is. portrayed. 

/' ' ^ 

e. " Validity . Since the :Various racial compositions 
of the Fick-A-Class Test are embedded within different class-' 
room activities on each page;; it is difficult for the student 
to infer which aspects of the, stimulus sketches are considered 
importfant by adults.. This partial disguise of the purpose of 
the test should reduce the likelihood that students will give 

For teacher pr'feference^ the student is scored as being 
either pro-white , • pro~b lack , of a.o' having no consistent 
racial preference . For classmate ^references, the student 
is scored as being either pro-white, pro-black, pro-mixed, 
or as havihg no consistent racial pteferencs. 

Since scores .OTi the Pa,ck-:A-Clas's Test constitute,^^ nominal 
rather than ordinal data , nonpar ame trie statistics ^ch as 
' chi--square are iJsed in analyses . 
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socially compliant, responses. Even if some students do infer 
the purpose of the test/ they are never' confronted with 
having to choose between two pictures varying only in racial 
composition: for those who prefer not to choose on racial 
grounds , there is. the, opportunity to respond in terms of the 
activities- portrayed, 

• / ' .; ^ ■ 

" ' The construct and predictive validity of the 
Pick-A-Class Test have been demons*trated in^several studies^ . 
l€ has been found thati students^ in schools with racially 
balanced, classes^ show Lignif icantly less racial polarization 
in their preferences for teachers, and somewhat less racial 
polarizatidn in their preferences for classmates, than stu- 
dents in schools with racially unbalanced classes • In a 
study of the effects of ciassroom racial hei-terogeneity on 
attitudes (Koslin, Koslin, and Pargament, 1971)^ it was 
found that black students in classes with 15%\or fewer blacks 
are more likely to express a preference for all black class- 
mates than are black students in classes wi-t:h about 50% 



blacks. Finally, in a study carried out to determine whether 

' - • . ■ / ... " 



. Koslin, S. , Koslin, B.L.', Pargament, RV", & Waxman, H. Class- 
room racial balance and students' interracial attitudes. 
Sociology of Educatio n, 1972, in press. . ' 

Koslin, S., Koslin, " B.L. , and Pargament, ^R. Relationships 
between educational, integration policies and students' racial 
attitudes. Presented at the meetings of the American Educa- 
tional Research Association, New York City , /February , 1971. 
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scores on an attitude test battery which included the Pick-A- 
Class. T(^st could predict sociometriq. choices , classmate and 
s^tfeacher. preference scores ^owed higher correlations (r = .28 

and r ~ .22) with the proportion of black peers nominated than 

• ' - ' . ' ' ■ ■.'^ ' 

did any other predictor- variable, . ^ 

Thus studies have shown that the Pxck-A-Ciass ; ^ 
Test is sensitive to differences in classroom racial balance 
and classroom racial heterogeneity, and that it predicts 
sociometric choices. 

f . Reliability . The Pick-A-Class Test was adminis- 
tered to approximately ^200 third grade students in a test- 

\ 

retest reliability study with a one-month interval between 
testings. The majority of students received exactly^ the 
same* score on the posttest as they had on the pretest, 
and only two percent shov/ed a reversal in classmate or teacher, 
preferences. These data' indicate that the Pick-A-Class Test 
is sufficiently reliable to detect differences betveen groups. 



For teacher preference, 70% of students *had identical scores 
on the pretest and on tlie posttest (x^ = 71.3, df = 4, £ < .001). 
For classmate preferences in comparisons with black teachers, 
58% of students received the identical score in the pretest as 
in the posttest (x_^ = 93. 9, df - 9, p < .001) ; for classmate 
preferences ..in comparisons with white teachers, 55% of students 
received identical scores^ on the pre- and posttests (x^ = 87.6, 
df = 9, £ < .001). Based on these data, r3iiability"T>oundaries 
for the teacherVpreference subtest are estimated as .57 <^£ < .75 
for classmate preferences with a black teacher, .^45 < £ <"^oo8; 
and for classmate preferences with a white teacher , .41" <.p < .64 
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and' thus is suitable for the types of between group analyses 
carried out in the evaluation of desegregation programs. 
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CHAPTER V ' ^ ; 

' ^ . . Evaluation Designs 

This chapter, reviews some of -the major design issues 
which school districts are likely to encounter in carrying 
' out studies to determine the cognitive and noncognitive ef- 
fects of desegregation programs. Some of the design issues 
discussed in this chapter are peculiar to the evaluation of 
desegiregation ^programs- 'and are reviewed iij detail because 
they may not be familiar to school district personnel. Other 
problems discussed are more classical issues of evaluation 
design which may be familiar to the reader but v/hich / be.cause 
of the special importance they assume in the context of de- 
segregation research, warrant mention here. > 

Solutjions to some of the problems discussed are likely 
to be beyond the capability and resources of local school 
authorities, and will require that the. district seek profes- 
sional assistance from the State Education Department (SED) 
or from qualified cojiisultants • ,It is_ the purpose of this 
chapter to signal these problems for school district personnel 
so that 'they will be able to . distinguish between problems 
which they can handle on their ov/n and those with which they 
will require assistance. Moreover, since the nature of the 
problems, e.g., statistical, administrative, etc. will be 



specified, districts should be able to define the particular 
types of consultative skills and expertise. which they will 
need. 

This chapter should also alert districts to .the possibil- 
ity.that -^here may be. one or more, problems o^ evaluation de- 
sign which, given the nature of the district and the program 
which is planned/ cannot readily be resolved completely even • 
with outside help, and which will therefore require that ef- 
fectiveness test data be interpreted cautiously. 

^. Organization of this chapt er. The heart^of this ^ chapter 

is the presentation' of alternate types' of evaluation designs 

■ ■ t 

> 

and a discus&ion of their strengths and \7eakn^sses. Before 
these are presented, a number of theoretical ajid-^proce^ural 
issues which cut across all designs are' discussed. Once the 
Concepts embodied, in these issues are understood, the specific 
strengths and weaknesses of the particular designs should be 
more easily grasped. 

In view of the multiplicity of existing and potential de~ 
segregatioji programs, no attempt will be made to present a 
comprehensive review of all possible evaluatioh designs. In- 
stead, a description of some principal design types and an 
analysis ''of their associated costs and benefits should suffice 
to sensitize 'readers to the major design issues. 

The various types of designs will be presented. sequeh- 
tially, from -y^e simpler to the more complex. However, in 



grouping the designs for a logically • ordered discussion, it 
will not be useful to employ a single simple-to-complex pro- 
gression, since desegregation programs which involve only a . 
portion of a district's students require a fundamentally dif- 
ferent design approach than programs where all students are- 
involv^d. Therefore, designs for the two types of situations 
"(all students involved versus. only some students involved) 
will be considered separately. While the actual design de- 
tails (and thus the information derived) differ between these 
two major classes of programs, it will be seen that the two 
types share a largely overlapping set of basic methodpJ.ogical 
concerns . 

it will further be seen that in desegregation research 
the simpler designs .usually have limited utility with respect 
to yielding definitive conclusions about the effects of a de- 
segregatiofp program and that, as increased design and ana- 
lytic complexity are added to the simpler plans, the informa- 
tion gained and clarity of interpretation increase. However, 
each gain derived from increased complexity usually comes at 
some type of cost to the district. • l^hese costs and ^nefits 
will be summarized at the end of the chapter. v/ 
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A • Issues Which Cut jikcross All Designs 

1*. Selection of independent and dependent variables > 
All designs to be , discussed share , a common purpose: to, assess 
the effects of varying selected environmental factors on one 
or more kinds of cognitive and noncognitive outcomes , given the 
context of a particular set of student and school characteris- 
tics. The care with which these various factors are represented 
in the research design wiir, in large measure, determine the in- 
-^erpret^ability and utility of the evaluation study, 

a. Environmental variables . The same environ- ■ 

t 

mental factors (e.g., racial balance, racial heterogeneity , 
etc.) which were treated as dependent data (outcomes) in the 
analysis of structural changes-during the first stage of eval- 
•uation are used as independent or "control" variables (i.e., 
as classif icatory meastires) in assessing the cognitive and 
noncognitive .effects of desegregation. 

The environmental characteristic directly al- 
tered by the desegregation process (classroom racial balance, \ 
school racial heterogeneity, or any other structural character-' 
is/tic) constitutes the principal independent variable. Using 
xthis variable to classify students enables the district to ask 
the question-; What consequence does a change in the level or 
value of this variable (e.g., balanced as opposed to unbalanced 
classes, high proportion of black students versus low propor- 
tion of black st:udents, in different schools etc .) have on 
cognitive or noncognitive outcomes? 
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To'^ Obtain an accurate answer concerning the ef- 
feet; of a desegregation program on cognitive and noncognitive 
outcomes, those balance and heterogeneity characteristics 
which are not affected by^ithe desegregation program but which 
nevertheless vary across educational environments must be ac- 
counted for in the design. Variation in these factors has^ 
either been shown by res'^arch (or may be presumecd on strong 
theoretical grounds) to affect cognitive or noncognitive out- 
comes. Consequently r the desegregation treatment may not have 
the .same effects under different environmental conditions, \ 
i.e.,- the effects of desegregation may interact or vary with 
the .surrounding conditions. 

Since the design should permit the district to 
determine whether the effects of desegregation>3^ary aS^a func- 
tion of other environmental conditions, it will b^ necessary 
to hold constant or otherwise^ control these other conditions • 
The pbject of such control procedures is to remove the effects 
of these other^ factors so that, as nearly as possible, the ef- 
fects of the structural change produced by the desegregation 
program can be assessed. 

When the balance and heterogeneity indices show 
that a particular factor has a constant (homogeneous) value in 
all environments , 'then that factor is, in effect, naturally "con 
trolled" and there is no reason to use it as an independent 
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or classificatory variable. For example, i& in a district 
where the desegregation objective is to change school racial 
heterogeneity all schools are found to have racially balanced 

. classes both before and after desegregation, there is no rjea-s 

• - . . * 

son to use classroom racial balance as a variable in the de- 
sign, Since classroom racial balance does not vary, it scannot. 
have any measurable effect, i,e', , it cannot differentially 
affect students' scores, U - 

However, if a balance or heterogeneity factor 
other than the one deliberately being alteir^d by the program 
' is found to vary across environments, it should be used^ as a 
way to classify students for the evaluation design. For ex- 
ample, in a desegregation program designed' to affect school 
racial balance, some students may be assigned to schools with 
racially .balanced classrooms, whereas others may be assigned , 
to schools with racially unbalanced classrooms , (This could 
occur if some schools in*^ a district practiced ability or 
achievement grouping whereas others did not.) Since classroom 

racial balance can affect students ' attitudes and achievement, 

i 

districts in which different schools vary widelly in degree of 
classroom racial balance should classify students according to' 
the level of classroom balance experienced, so that comparisons 
can be made to determine the effects of different levels of . 
school racial balance under racially balanced versus racially 
unbalanced classroom conditions. 



To take another example: if the objective of 

the desegregatibn program is to alter classroom racial balance 

••■.• „ . . . . 

within schools, and if there happens to be appreciable varia- 
tion between schools with respect to racial heterogeneity 
(i.e.-., the schools are unbalanced) , this heterogeneity" factor . 
should be used as a variable in the design, so that it will be 
possible to detect and compare differences in the cognitive or 
^noncognitive effects of altering classroom racial balance when 
the proportion of black students is low versus when it is high. 

" Another important consideration in the selection 

of independent and control factors is the possibility that a 
change in one environmental variable mi^y be accompanied by a 
change in a second variable*. When two or more variables are 
correlated with each other, and hence change simultaneously, 
it is difficult to interpret either variable as a fully inde- 
pendent factor in the design. - . 

For example, because of the correlation in pur. 
society between race and social classr (SES) / a planned change 
in racial balance i^ often accompanied hy a change (planned or 
unplanned) in SES balance. ,Since it is vddely recognized that 
SES influences achievement ^ the correlation between the raca 
and SES composition of educational units make's it difficult 
to assess the specific cognitive or noncognitive effects at-"^ 
tributable to a change in racial composition. 



Because of the strong face-SES" correlation in 
the general population, most school districts cannot reason- 
ably expect to disentangl^.^the^ effects of changes in SES 
milieu from those^^df liihanges in racial composition/ , However/;^, 
in some districts it may be possible to desigrr^studies so that 
racial com.position and SES composition are distinguishable ^s . 
factors,, in order that the relative influence of changes in 
racial versus SES composition ban be evaluated* This could 
happen if so^ie black students were randomly assigned to high 
SES schools and others were randomjly assigned to. low SES 
schools. Comparisons between blacks assigned to" classes or 

schools differing in SES composition could" provide informa-^ 

■ , \ ■ ■ ■ . ■ 

■ ■ ■ . \ ■; • ■ • 

tion on the effects of SES desegregation* as compared with the. 

■ ^- 

effects -of racial desegregation (proyided that the groups of 
blacks assigned to the two s '-nools were initially • equivalent 
and that curriculum, teachers; special services, etc. were ^ 
comparable in the high and low SES "schools) . 1. 

As was pointed out iii Chapter 11^ various proc- 
esses may be used to i^mplement de'segregation. The effects of. 
desegregation may vary according, to the process used. - In se- 
lecting environmental , factors to serve as independent or con-, 
trol variables in an evaluation design, districts should 'treat 
implementation processes as a type of environmental variable 
and should select a design in which various implementation 



processes can distinguished whenever several- processes are 
being usGsd simultaneously. . . ' 

For example^, in implementing desegregation, the 
district may assign some black students to schools within walk- 
ing distance of 'their hoit^es whereas others may/ be bused to 
another part of the district. Under such circuaustances , pro- 
gram effects may differ between bused and non-bused students 
since r for example / opportunities for casual emd Informal in- 
teraction between the races will probably be fewer for students 
who are bused than for those who are not; hence interracial at- 
titudes may differ for the two groups of black students • In 
this example, mode of transportation should be considered as 
an independent or control variable, so as to perndt comparisons 
to determine whether program effects are different for walkers 
versus riders . v 

School districts may also find that certain en- 
vironmental variables, are correlate^l, e.g., that variation in 
classroom racial balance tends to be associated with variation 
in proportion of black students, or that proportion of black 
students tends to be "associcited with busing versus walking", etc 
Under. such conditions the variables are not fully independent 
of each other (i.e., they are "confounded"), cind unambiguous 
data will be hard to obtain. A consultant with considerable 
skill in experimental design and statistical smalysis will be 
required to suggest a design which will make it possible to 
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separate and thus to assess, the effects of these empirically 
confounded (correlated) factors, at a cost that the district 
can^; afford. 

b» Personal variables . A number of personal stu- 
dent characteristics should also be used as variables in the 
design, sincle the effects of the^-desegregation program may 
differ for various subsets of the student body. For example , 
the program' may not affect blacks ^d whites similarly, or may 
have a different eff:ect on high versus'^ low SES students, etc. . 
The evaluation design should enable the district to assess any 
such differential program impact. The most frequently used 
personal variables for classifying^ students are: .^race^, age 
(grade), sex, IQ,' achievement level, and SES background. The 
more detailed the conclusions which a district .wishes to draw 
about program effects on different groups of students, the 
greater the number of personal'^variables which must be repire- 
rented in the design. * \ ' *^ 

2. Independent variables must be limited in number . .The 
discussion to this pbint has illustrated' the need for careful 
review of all the environmental and personal variables which 
•,might affect the outcomes of desegregation, and has, pointed out 
that a district's ability to draw precise cpnclusions will de- 
pend on the selection of appropriate independent variables for 
the evaluation study. • . . 
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'However, no district is likely to be able to treat 
all relevant factors as independent variables. First, an eval- 
uation study rapidly becomes both administratively unmanageable 
as well as prohibitively expensive as independent variables are 
added, because . the sample size required for re,liable data . in-- 
creases geometrically as th^ number of independent variables 
or the number of levels of the independent variables is in- 
creased. For example, increasing the number of grade levels . 
tested f2?om one to two doubles the size of the required sample; 
treating both classroom racial balance and classroom racial 

heterogeneity as independent variables requires twice the sam- 

/ ■ ■ ■ ■ / 

pie size needed when .only one of -bhese variables is used. . 

^ Second r the^ confounding of certain variables (as il- 

lustrated above) creates statistical problems in designs , since 
not all possible combinations • of the independent variables and, 
their levels will actually tie Eound in the district. For ex- 
ample , racially balanced classrooms may be found only at some 
levels of racial heterogeneity, rather than at all levels. Un- 
der these circumstances, only one or the other of the. balance 
and heterogeneity ' factors may be used as an independent varia- 
ble. : 

Finally, the sheer .analytic complexity of data from 
designs with more than four independent v;ariables probably ex- 
ceeds the available interpretive capabi^lity of most districts 
and of most consultants. Working with- a statistical consultant 
district' personnel should therefore review the environmental 
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and personal variables which are particularly important in 
their district, in order to select the major program treatments 
which must be used as independent variables and to identify 
other important factors which can be treated as control varia- 
bles via any of several sampling 91: statistical procedures 
(e.g., nested designs, replicate designs, etcc). 

3. Dependent variables . In the assessment of cogijitive 
and noncognitive effects . of desegregation, 'the dependent varia- 
bles are scores on those^ tests selected by the district. to 
serve as effectiveness measures , e . g . , attitude test scores, 
reading test scores , arithmetic test scores) etc. 

' . Each additional dependent variable included in the 
evaluation " study increases the variety of cognitive or noncog- 
nitive outcomes assessed, but alsr'o adds to the costs -of the 
study with respect to test purchase., length of testing session, 
scoring of booklets or answer sheets, computer time, and inter- 
pretation of 'results. However, in contrast with the costs of 
increasing^ the number of independent variables or their levels, 
the number of dependent variables does not affect sample size 
or characteristics . . 

4 . Advance seleption of variables and planning of de- 
signs . It was noted dn^ Chapter II that if analysis shows that 
the desired structural changes in student body composition 
have not taken place, the district should not proceed to 
assess cognitive or noncognitive program effects, since the 
planneC desegregation .has not actually occurred. 



Despite this^need to verify the structural conse- 

, se- 
quences of desegregation programs before proceeding to assess 

cognitive and noncognitive outcomes^ the, design for the latter 
study should be decided upon before desegregation ds begun. . 
The design chosen will determine whether or not students need 
to be tested before desegregation ip implemented* ^nd, if so, 
who will be tested. Specification of the design may also in- 
fluence assignment policies, so that students can be retained 
in classifications needed to complete the study For example, 
if level of racial heterogeneity is a control variable, stu- 
dents should be assigned so that they are in the same level 
after deseareaation as before. 

Therefore, it will usually be preferable to fully 
design the evaluatipn study ahead of time,- rather than to ^defer 
design decisions until the extent of the structural change 
brought about by desegregation has been determined. If neces- 
sary, elements of the design op analytic procedures can be 
modified on the basis of unanticipated changes shown by the 
balance and heterogeneity indicators. 

5. Sample size . The nxomber of students to be tested, ' 
as well als their characteristics , will vary as a function of 
' che independent variables selected for inclusion in the design, 

* The discussion, below, will^ show- that most types of designs cfc 
require testing prior to desegreg'ation. 
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the size of the district, alnd the financial resources available 
for the. study. However, for any design there will be a minimum 
"sample size required. Below' this minimum sample size, fetxere 
would be too few students representing the levels and combina- 
tiofts of the independent variables to yield Stable data. There- 
fore, the school district should work with its statistical con- 
sultant to^ determine what the minimum number of students of 
various characteristics must be to permit the desired analyses - 
to be performed.. ^ 

Where resources are limited, and there is consequently 
a ceiling on the total number of children who may par^ticipate in 
the study, the district must be especially careful not to spread 
the sample across so many grade levels tha^ there will^e an in- 
adequate number of students in each. While it may desirable 
to test at more than one grade level so that the effects of de- 
segregation at different ages can be assessed, it will not be 
worthwhile to do so if the resulting number of students in each 
grade will be too small to yield stable data. It would be bet^ 
ter to concentrate the sample in fewer grades to be sure of hav- 
ing stable data for those grades. 

' 6 ; Sampling an adequate number of .minority students . In 
most districts, school authorities will want to evaluate and 
compare the, effects bf. desegregation on minority and on majority 
children. In districts where schools have only a small proper-' 
tion of minority students, a larger number of classes will need 
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to be tested to obtain reliable duta for minority students 
than would be required where the minority comprisesi a greater 
proportion of the student body. 

• Assuring that samples contain adequate numbers of 

minority students for purposes of analysis will be- especially 
hard if these students are evenly distributed among "^classes • 
For example, if there are only one or two black students per 
class a great, many classes must be tested to obtain enough 

black students ^o yield stable data. When a district's evalu- \ 

' ' ' ' . ■ ■. / 

ation budget is limited, such massive testing may not appear 

.. • ' \ . . ■■ ■ • . ' ■ 

worthwhile, since testing a large number of, intact classes 

will yield far mo^e data on white students than would minimally 

be required for carrying out analyses . 

However, alternative ways of securing data from an 

adequate number of minority students pose drawbacks of 

their. own. For example, if minority students are called to a 

special room^ for testing, or are sent to take the test in a 

class other than their own, it will probably not be valid to 

compare the results of their tests with those of white students 

who^ ar,e tested in their regular classrooms. Alternatively, if 

approximately equal nuntbers of blacks and whites are called to 

an auditorium, the novelty /of being singled out for testing 

and taking, the test(s) in a half-v;hite, .half-black setting 

casts doubt on test validity, especially the validity of racial 




attitude data. Finally, if the problem' of too few blacks per 
class is resolved by restricting testing to only those schools 
or classes where blacks comprise a larger minority, the gener- 

I , ■ 

alizability of the data to the district as a whole is jeopard- 
ized. .V 

Therefore , districts which have a small proportion - 
of black students in some (or all) schools, and which also 
have limited evaluation funds, will^ need to consider reducing 
either the number of different kinds of tests given or the 
number of grades tested, in order to sample enough classes 
within selected grades to insure at least some stable data 
concerning the program's effects on minority students. 

7 . Accurate identification of student characteristics . . 
In order to assess any differential effects which a desegrega- 
tion program may have on majority and minority students, it is 
esseatial tha^t the race or ethnicity of students be identified. 
Accurately. Whereas for sample selection purposes it is ade- 
quate to know merely the proportion of minority students in a 
school or class, for analytic purposes the racial or ethnic 
characteristics of each and every student tested must be ac- 
curately recorded. If SES .is to be used as a Variable in the' 
evaluation design, it too must be Accurately re\corded for every 



student. It is usually a good idea\ to secure this information 



prior to the time of testing, especj\ally when testing is sched- 
uled for the end of the academic year. Provision must also be 
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made for accurately matching all the- information identifying 
each child with each .test he will t'ake. 

^» Stability of the desegregation program . /Assessment 
cf the cognitive and noncognitive effects of a desegregation 
program requires that the, program not be changed in any signif- 
icant way during the course of the planned evaluation, i.e., 

! 

that no change take place in such important elements as the 
program objectives, the means of program implementation, etc. 
Since all the designs discussed btelow assume that the major 
independent and control variables and their levels are'^d.onstant 
throughout the study, any, change in these factors will invali- 
date the planned analysis of data. 

In the event that there is a compelling reason for 
the district to make a major change in orie or more program 
element (sj during the course of the study period, the evalua- 
tion design will need to be thoroughly reviewed. and /either 
altered or replaced by a new design. 

9. Time of testing . Late spring is the preferred test- ^ 
ing time for evaluating desegregation programs because, by 
that time, students will have had maximum exposure to and^ ex- 
perience xn the desegregated environment. ISpecial research 

situations\ may ' require districts to test .at other tiimes .during 

' ■■ \ J 
the year (s\uch as where a spring pretest was not feasible 
\ - ■ ■ , . • - ■ ■ ■ ■ ■ 

prior to th^ start of a-program, and hence an early fall test- 
ing was substituted) , but wherever possible — and especially 
when repeated\ testing is planned on a yearly basis — -testing / 
late in the academic year is to be preferred. 
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B. Designs for Desegregation. Programs in Which Only Some 
. Students are Program Participants 

The effects of desegregation* pr.ograms in which only some 
(rather than all) of a district's students are involved may 
be assessed by comparing the test scores of desegregated btu- 
dents (participants) with the test scores of nondesegregated ■ 
students (nonparticipants) . Differences in scores between 
the two groups are presumed to be due to the effects^ deseg- 
regation. Such designs are based on the assumption that a 
suitable group of nonparticipants is available for test score 
comparisons with participants. 

There are two main -variants of such designs . In the 
simpler form, participants' scores are compared with non- 
participants' scores only after the desegregation program has 
been in operation for a specified period of time. In the more 
complex form, participants* scores are compared with those of 

nonparticipants both prior to desegregation and following , 

• ' .. ■ ■ ' ^ ■ . 

' ■ ■ - ■ ' (/ 

desegregation. 0 

In discussing these designs and in illustrating them ^ 
schematically, time^ t^ will refer to that test administration 
which takes place prior to the implementation of desegrega- 
tion, and time t2 will refer to that tes^t administration which 
takes place following thq implementation of desegregation. 
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1. Type I Designib; Postdesegregation comparisons only . 
In Type I designs, the desegregation program is evaluated by 
comparing the test scores of program participanrs and non- 
participants after the prograin has been in effect for a speci- 
/ . f led period of time. There. is no testing of students prior to 
implementation of desegregation. 

A Type I design is shown schematically in Figure; V-1 
' The tj^ row indicates tli^at no testing occurs prior to the imple 
mentation of desegregation; the t2 row indicates that testing 
occurs for both grpups following implementation of the program 
The' critical analytic .comparison in this evaluation design is 
between the scoir^es of participants and nonparticipants ♦ 
(comparison a in Fig. V-1) . . 

a. Advantages of Type I designs . . Type I designs 
have thd advantage of requiring only a single test administra- 
tion, thereby minimizing administrative costs of 'testing, scor 
ing, etc. There is also no need to keep track of individual - 
students over a period of time, thereby reducing clerical 
(record-keeping) requirements. ' , / 

b. • Disadvantages of Type I designs . The chief dis 
advantage - of - Type I designs is that the data which they yield 

\ are usually uninterpretable. In" comparing, only the t,^ scores 
of program participants and nonparticipants, 'any t2 differ- 
ences between the groups cannot unambiguously be interpreted 

■ ■ ' ' ■ ■ ■ " 
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Fig. V~l. A Type I Design: Letter in (). indicates , 
comparison referred to in text. 
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as being due to desegregation^ Since t^ere is no adequate 
way of ruling out initial differences which may have existed 
between the two groups. 

While the possibility of initial nonequivalence ,r -v:^: . 
between program participants and nonparticipants is a common 
one in the evaluation of educational programs, it poses an 
especially serious problem in desegregation' research because 
the children undergoing desegregation frequently are a dis- 
tinctly nonrandom sample of all students in the districts 
When a desegregation program does • not. involve all the children 
in a school or school district, it is often the case that par- 
ticipants have volunteered for the program (or have at least 
agreed to cooperate with it) . Whenever this situation occurs, - 
the possibility exists that the attitudes and achievement of ' 
participating students may be different from those of students 
who have not volunteered or have not been williaig to partici-^ 
pate. Consequently, if differences are found between the test 
scores of participants and nonparticipants after the program 
has been in operation for a .specified period of time, i,t is 
not legitimate to conclude that, the differen'ces are due t'o the 
effects of the program: except where assignment to the -program 
has been completely random, there is always the possibility 

V' _ _ - - •' / - - 

that: the groups were not fully comparable at the start. - 

The fundcimental problem in Type redesigns of uncer- 
tainty concerning the initial comparabi-lity of participants and 



nonparticipants is so serious {unless initial assignment was - 
random) .that these designs seldom yi'qld definitive results . 
Consequently, Type I designs should be avoided. . . 

However r in those instances where a design of Type I 
is the only design possible for a school district , a-number of 
steps can be taken to increase the interpretability ofthe t2 
data. The scores of program participants should, not be com- 
pared with a randomly, s^elected group of ^nonparticipants , but 
rather should be compared with the scores of a group of non- 
participants matched to the program participants on as many 

important characteristics as possible. 

• ■ ■■ 

The groups should be. matched on the following per- 
sonal variables: race.,,, age, sex, previous academic a'chieve- 
ment, and family background. Matching on environmental varia- 
bles not affected by the desegregation program should also be 
attempted, e.g., matching on school or classroom balance 
characteristics which are not altered as part of the "program. 
The matching may ^be based either on i^ndividual scores ( "matched 
pairs") or on group means; however, \ the two methods require 
different types of statistical treatment. ' " 

Another good way of matching would be to use a com- 
parison group composed of students who volunteered for the 
program but who could not be accommodated due to a limited 
number of openings. This procedure effectively matches, par- 
ticipants and nonparticipants on the attitudes toward school, 
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desegregation, achievement, etc.^ which notivated students to 
volunteer to participate in the program. 

2. , Type II Designs; PrecLesegregation and postdesegre- 
gation .comparisons * In Typ® designs, program participants 
and nonparticipants are tested before the desegregation pro- 
greau,; begins and are tested after the program has been in 
operation for s specified period of time. ^Type II designs 

" ^ ' ■ V-.. 

are illustrated in Figure V-2, where it can be seen that ithe 
spaces for participants' and for nonparticipants are filled 
both at t, and at-t^. ^ 

• A difference between program participants and non- 
participants in the degree to which test scores. change from 

t^ to ^2 (a versus b in Fig. V-2) or, alternatively, a differ- - 

. . ' ■'S, , . 

ence in scores between program participants and nonpar tici- 

pants at t^ when there was no such difference at t, (c versus 

d in Fig. V-2) is interpreted as being due to 'the desegregation 

program. " 

a. Advantac^e of Type II designs » The advantage 

of Type II designs, when qompared with Type I designs, is> that 

it^ is possible to determine whether participants ,and non- 

participants were matched (i .e. , had equivalent scores) on 

the dependent (effectiveness) measures prior to the inception 

of the desegregation program, and whether they wore comparable 

on important personal and environmental characteristics. With 

such information, a determination can be made concerning 
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Fig. V~2. A\ "^yP^ Design: Letters in ( ) indicate 
comparisons referred to in text. 



whether or not the nonparticipants. constitute an appropriate 
group, with Which to compare participants ' t2 scores. There- 
^ fore Type II designs result in data which are more interpreta- 
ble than those' obtained from Type I designs. 

b. Disadvantages and probliems of Type IJ designs . 
Type II designs are more .expensive to complete than are Type I 
designs. The addition of a pretest doubles testing and scor- 
ing costs, and raises' analytic (i.e.. computer arid interpre- 
tive) • cpsts because the number of statistical cpmparisons 
which must be made is increased. ^ / 

Type II designs also require greater adminis- 

. / ■ 

trative capability than Type I designso First, districts must 
have their desegregation plans sufficiently detailed in ad- 
vance to know which students will participate and which stu- 
dents will not. Second; districts must^be^^-sufficiently or- 
' ganized to complete the process of s^decting and testing stu- 
dents prior to commencing the desegregation program, despite 
the fact that pressures for evaluation within the district are 
^'^"-^"^^''^"'^^ to be low at that time= Third, assignment of students 

to classes and schools must be made systematically, so that 



int/o 



■ ^\ ■ 

students will fall into the same levels of the independent and 

control variables for the posttest that they were in for the 

pretest. Fourth, the pretest sample must be selected with a 

feasible posttest schedule in mind^. since students who are 

grouped together physically when tlie pretest is administered 

might otherwise be widely scattered when it is time for the 



posttest. Finally, record keeping must be accurate* Pretest 
data for each student must be stored without error and in such . 
a way that the t^ scores can be readily retrieved and matched 
with the t^ data. 

Problems may be encountered regarding t-.he equiv- 
alence of the t^ arid participant and nonparticipant groups. 
Districts must be alert to these problems since, if they occur 
and are not appropriately dealt with, the advantage of pretest- 
ing can be entirely lost. 

The following problem might occur in some dis- 
tricts. The group selected to serve as nonparticipants may turn 
out to be , an inadequate control group. For example, self- 
selection processes among students who volunteer to participate 
in the desegregation program may result in significant differ- 
ences in the initial (tj^) attitude or achievement scores of 
participants and nonparticipants. If this nonequivalence is 
discovered early enough, additional sampling or a modification 
of the sampling plan might be feasible. Otherwise, it will be 
necessary to make use of analytic procedures which statisti- 
cally "equate" the groups for their initial difference in scores. 

Another problem may^occur if students drop out 
of the program in a nonrandom pattern. This situation may 
result in the groups of participants tested at t^^ and t2 being 
different from each other. For example> students who drop out 
o£ the program between t^ and t2 may be those who have the most 
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negative attitudes toward their school or their new teachers 
andoclassmates , or' those whose academic performance is poorest. 

^ Thus students who remain in the program for testing at t2 may 
represent a select subset of the original group. Since program 
participants and nonparticipants were matched for equivalence 
at t^, and since it is assumed that nonparticipants are less 
likely to "drop out" than participants, the net result of selec- 
tive dropping-out processes is apt to be nonequivalence of par- 
ticipants and nonparticipants at t2« If this occurs, the ad- 
vantage of a control group, obtained- by, matching participants 

* and nonparticipants on the basis of the pretest, is lost. 

Therefore, prior to the collection of t2 test 
data, those students who have dropped out during the course of ' 
the evaluation study should be statistically eliminated from 
the tj^ sample, and the consequences of this deletion should be 
analyzed. If it is found that delation of the dropouts destroys 
the initial (t^) equivalence of participants and nonparticipants, 
it will be necessary statistically to eliminate some students 
from the nonparticipant group so as to restore the £^ equivalence 
of participants and nonparticipants . If statistical equivalence 
cannot be restored, the district will have difficulty interpret- 
ing the t2 data, and should seek the help of a consultant before 

proceeding. \ - 

Moreover, if analysis of the\data reveals a. sys- 
tematic difference between program dropouts and non-dropouts witii 
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respect to# scores or with respect to personal or environ- 
mental- characteristics , these facts should be noted, as they 

^ ,. * 

represent important findings in *their own right • 

Another potential source of difficulty in 
Type II designs lies in the fact that the reliability of a 
difference score is always lower than the reliability of. the 
test itself.. This means that any difference or change in 
score between t^ and will be less reliable than either the 
or the t2 test scores themselves. Because of the relative 
unreliability of individual change scores, districts should 
plan to employ statistical procedures which analyze the aver- 
age differences between the scores of the groups of program 
participants and nonparticipants at t. and at t^ . Several 
methods exist for carrying out such analyses, and districts 
should obtain the assistance of a statistician to select an 
appropriate one. Extreme caution must be exercised if dis- 
tricts decide to compare changes In the individual scores of 
participants with changes in -dhe individual scores of non- 
participants, especially if the sample size is small, or if the 
reliability of the tests used is not very high. ' 

Assuming that all of the methodological prob-- 
lems discussed above have been satisfactorily resolved> and 
that the design is therefore basically sound, some district's 
may nevertheless experience uncertainty in interpreting the 
results of their study if participation in the desegregation 

- ^ 
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program is confounded ^(coincides) with other educational changes. 

In many districts, desegregation is accompanied 
by., and thus confounded with, other new or experimental programs 

■ C ■ ' ■ ■ V 

which might themselves account for changes in scores on the 
effectiveness measures. For example, the introduction of study 
units on inter.cultural relations may affect racial attitudes; 
the provision of tutors, specialists, or reduced class size may. 
affect achievement and attitudes ; changes in the content of the 
academic curricul\im (e.g., a new reading or math program) may 
result in test itpnis on the posttest being better matched to 
curricurum content for program participants than for -non- 
participants, or vice-versa. Under any of these conditions, 
posttest differences between the attitude or achieve ment scores 
of program participants and nonparticipants cannot be attrib- 
uted with certainty to the change in racial composition. 

In another common situation, desegregation is 
confounded with attendance at better quality schools. Deseg- 
regation programs frequently result in minority children from 
schools of low quality in poor neighborhoods being transferred! 
to better schools in another neighborhood within their own 
school district or, in some cases, across district lines. The 
schools to which children are transferred may have better 
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facilities, more talented teachers, smaller classes, etc. than 
the schools previously attended by =-4:hese students (in fact, 
the provision of "bettf^r quality education" is often the pub- 
licly stated reason for initiating the desegregation program) . 
Under these conditions, it is possible^, that changes in the at- 
titudes or achievement of desegregated students are due, at 
least in part, to the better qualrty education offered in the 
new school. in such a case, posttest score differences be- 
tween program participants and nonparticipants cannot be at- 
tributed directly to the change in racial composition of the 
school attended. 

As noted earlier in this chapter, because race 
and socioeconomic status (SES) are highly correlated in the. 
United States, the desegregation process often results in a 
change in the social class characteristics of a child's class- 
mates. Typically f desegregation results in an increase in the 
average sO/Cioeconomic status of' the classmates of black stu- 
dents^ and a decrease in the average socioeconomic" status of" 
the classmates of white students. Since it has been demon- 
strated that the social class characteristics of a black stu- 
dent's classmates . affect his academic achievement regardless 
of his ov/n SES, it is possible that significant posttest dif- 
ferences in attitude or achievement scores between initially 
equivalent program participants and nonparticipants are due, 
at least in part, to the change in social class milieu. Thus 
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because of the confounding of race and SES, t2^dif f erences in 

attitude or achievement scores'" cannot , in most instances, be 

• . - ■ v 

attributed specifically ,to the change in racial composition* 

It must be stressed £hat the types of confound- 
ing discussed- in the preceding three illustrations may actu- « 
ally .be highly desirable from a pedagogical * point of view, in 
that they may facilitate desired changes in student perform- 
ance, e^.g., more favorable, attitudes , better achievement, etc. 
The fact that these sources of confounding are classified as 
"problems" is not. meant in any way to suggest that schools 
should refrain from introducing other program, personnel, or 
facilities changes as part of., or in conjunction with, deseg- 
regation. Moreover, the discussion above certainly doeis not 
imply that districts should refrain from evaluating a desegre- 
gation program if that program is multif aceted. ' Carefully 
designed evaluati'on studies should be able to provide districts 
with at least some useful data on the . overall effects of mixed 
desegregation programs. 

However, confounds of the type described above 
do present interpretive problems because they limit the in- 
ferences which may be drawn from the studies in which they occur. 
The more that changes in racial composition are correlated with 
other changes which could affect students* scores on effective- 
ness measures, the more difficult it will be to attribute 
observed differences in test performance to changes in racial 
composition. . , 
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The ability to attribute program effects specif- 
ically to racial composition is valuable because it gives the 
district an empirical basis for either continuing or changing 
a desegregation program. From a logical .point of view, if the 
program was planned to continue unchanged indefinitely, it 
would be of- little practical importance t^o disentangle the ef- 
fects of confounded program elements. But to the extent that 
the district may, in the course of time, wish to modify some 
elementXs) Of the program, it would be useful to know the spe- 
cific effects contributed. by the various factors, e-g,, what 
effects are attributable to particular racial composition con- 
figurations? ^ 

3, Type IIA Designs; Predesegregation and postdesegre- 
gation comparisons with random student assignment . Type. IIA 
designs are like Type II designs except that assignment of 
students as participants and nonparticipants is completely 
random. No self selection (volunteering) for the desegrega- 
tion program is permitted. In principle, Type IIA designs are 
to be preferred over Type II designs becau^f^ nonparticipants 
should represent a. more adequate control group for participants 
However, completely random assignment is seldom feasible in 
view of the variety of constraints (e.g. , political, adminis- 
trative, etc..) within which school districts operate. More- 
over, while random assignment may reduce control group problems 



the other sources of confounding discussed in connection with 
T^ype II designs apply equally to designs of Type IIA. 



/ 

/■ 
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G. Types of Designs When All &tudents are Prograin Participants 

In the previous section of this chapter, the designs dis- 
cussed (Types I. and II) were based on the assumption that only 
s,anie portion of the district's' students were participating in 
the desegregation prograin. It was therefore possible to make 
use of a contemporaneous control group in those designs. How~ 
ever, in districts where the desegregation program includes 
all students (or at least all those in a given grade) , a con- 
temporaneous control group does not exist. 

Since some form of control group is required for adequate 
' assessment of program effectiveness, districts where all stu- 
dents are involved in the desegregation program must make use . 
6f designs with historical or longitudinal controls ^ i.e., 
designs ' employing predesegregation scores for control purposes. 
The predesegregation scores can be either the child's own 
scores prior to desegregation (Type III designs) or can be 
those of same-aged children in an earlier year, before' the de- 
segregation program was insti'(:uted (Type IV designs) . Finally, 
it is possible to use both the child's own previous scores as 
well as the scores of other' children in . a single design (Type^V 
designs), thereby incorpDrating the advantages of designs III 
and IV while eliminating some of the particular limitations 
associated with either design alone. 

To keep the notation as consistent as possible with the 
notation used in the previous sections t, will refer to the 
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last testing session which precedes desegregation and t2 will 
refer to the first testing session which follows desegregation. . 
Where a design is expanded to include more than a single. pre 
and posttest, the progressively earlier testings will be 
indicated by increasingly smaller subscripts of t (e^.g.^ t^ , 
t_^^ , . . . t_^) , whereas the progressively later testings will be 
indicated by increasingly larger subscripts (e.g., t^f t^,... 

1 . Type III Designs; Longitudinal comparisons of scores 
for the same group of students > In Type HI designs (Figure V-3) 
students' scores before desegregation are compared with their ^ 
sqpres following desegregation. For example, scores of students 
in gradp- X before d ^segregation are compared with subsequent 
scores of the same students in grade X+1 following desegregation 
(comparison a in Fig. V--3) . It is assumed that changes in test ^ 
scores from pre to postdesegregation are attributable to the 
desegregation program. 

a. . Advantage of Type III designs . The advantage, of 
Type, III designs is that since exactly the same students 
participate in successive testings, there should be no question 
concerning the comparability of predesegregation and post- 
desegregation students on such important variables as race, seX/ 
SES, ^nd prior academic achievement , since each inaividual serves 
as his own control. 
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b. Disadvantages and problems of Type III designs > 
Designs of Type III share most of the administrative problems ' 
of Type II designs. > If the same students are to be tested on ^ 
various occasions and^\ their scores compared, then' assignments 

to classes aiid schools must be made in such a way, that the en- 

■■■ . ■ : . ■ \ ■ • ■ • . . - . 

vironmental factors which^are being used as control variables 

are held constant over time for each student* At: a practical 

level/ thought must be given to how best to sample and 4:o 

assign students so that iater retesting is feasible. Finally, 

^ ^ , - • .. ■• 

since the same group of children is tested repeatedly ,;'data 

. must be stored in such a' way that the previous scores of every 

student can be readily retrieved and accurately merged with 

new data. 

A special administrative problem of Type III 

designs -is the possibility of student attrition over the course 

of the study. In many distriotc the rate of student turnover > 

during the period Of the study will be high. Even if tliose who 

\ » ■ . ': 

A ■ - ■ 

leave constitute a random sample of the entire student body 

(e.g., they have not left because their parents disapproved of 

■ ° ■ ■ ■ ■ \ ■ " ■ ^ \ 

the new desegregation policy) , prior experience, with longitudinal 

J studies has shown that so large a proportion of ^e original 

sample may be lost over time that the remaininof^ sample may be 

too small to obtain stable findings . 

The l,onger^ the study continues, the greater the 

likelihood of student attrition-. For this reason, districts 
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undertaking longitudinal studies which require repeated testing 
of the same group of students s.houl^ plan to oversample (i.e., 
to test more students than will ac(tually be needed for later 
analyses) in order to insure that \here will/be an adequate 



number of students remaining in the' sample at the ei^ of. the 
study period. The oversampling should; be proportional to the 
known rate of student turnover in the district and to the 
anticipated duration of the study, but at least a ten percent 
oversampling should be, planned. ^ 

, Another disadvantage of Type II^I designs is that 
they have limited interpretability . There, is often' no way to 
demonstrate that th&- observed test. score changes would not 
have occurred without the desegregation program, since several 
factors which could be responsible for the changes in test 
scores from t^^ to t2 are fully confounded with desegregation 
in Type III designs. Thus it is hard to determine whether 
changes in test scores are due to the desegregation program. 

The most seriouS' problem in interpreting data 
from Type ill designs is that desegregation is confounded with 
maturation. Thus it_iaay-^e^argued that an observed change in 
score is maturational , i.e., that it is a" change which normally 
accompanies development between certain ages, and one which • 
would therefore have occurred even without desegregation. The 
longer the duration of the evaluation study, the more serious 
the problem of unknown maturational effects may become. 

■ ■ ■ ' 
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As the discussion in Chapter IV indicated, inter- 
pretation of scores on norm referenced achievement tests, on 
criterion referenced achievement tests > and. on racial attitude 
tests requires baseline data from within the district (local 
norms or control groups). in order to properly evaluate changes 
in test scores following desegregation. Without such baseline ^ 
data, observed changes, in test scores cannot be compared with j 
expected maturational changes in scoires for district students \ 
over the time period studied. Since Type III designs do not 
include a contemporaneous control group and do not provide 
for comparisons with predesegre^gation baseline data for same- 
aged students, it is not possible to rule out maturatioh as 
the source of changes in scores on these tests. 

Another difficulty in concluding with certainty 
that changes- in test scores are due to desegregation is that a 
change in score may arise from f amiliarityxwith the tests. If 
the same tests are used at both t ^^'y an^ ^2' it "is^ 
at least some part of the change iri scores is due to^^^tBre^-—-^ v 
children's prior experience in taking the tests. Since Type II 
designs include no students who have taken the tests^but who 
have. not been desegregated, changes^an score' associated with 
.repeated test-taking cannot be assessed. 



/ 

Following desegregation, a nximber of changes lua^ 
take place within the schools and within the district which/ 
on their own, could be responsible for any observed changes in 
students ' attitude or achievement scores. The occurrence of 
these changes in the district makes it difficult to attribute 
changes in effectiveness test scores to the desegregation pro- 
gram. . 

^ 'For example, the change in >test scores m'ay be due ^ 

to a change in the composition of the teaching faculty. When 
a school or school district undergoes desegregation, the pool 
of teachers may change. Teachers who have negative' attitudes 
toward teaching minority students may leave and be replaced by, 
teachers who are more motivated to work in an integrated setting 
and who, therefore, may tend to be more effective with racially 
mixed classes. Over a relatively short period of time, con- 
side'rable transformation of the teaching faculty cduld occur 
via this process. Under such circumstances, it would be difficult 
to conclude that changes over time in attitude or achievement 
scores were directly due to the desegregation program. For this. - 
reason, careful attention Should be paid to data on teacher 
turnover. In districts where such turnover has occurred to any 
marked extent, the change in teaching staff should be noted as 
an important outcome of desegregation in its own right, and 
should be cited as a possible factor in producing the observed 
changes in pupils' attitudes or achievement. 



There are several other school factors, discussed / 
■ ■ ■ . . ■ . / 

above in connection with Type II designs , which of ten accompany// 

desegregation and which could result in a predesegregation tp^ 

■ \ / 

postdesegregation change in the test scores of students • ^Briefly 

1. The general quality of schooling may improve 
as a result of the hiring of specialist?;, reduction of class 
sizes, new facilities, etc.; 

/ , • • 

2. Other new programs may be introduced along 

with desegregation, e.g., cxarriculiom units designed to improve 
intercultural relations, new reading programs, etc.; 

3. The SES composition of classrooms may change 
as a correlate of the chaxtge in racial composition. 

Finally, in Type III designs it is difficult to 
rule out the possibility that the change in test scores may be 
due to influences external to the school. Since the schools 
are not insulated from the outside world, changes in student 
performance may reflect reaction to outside events rather than 
to school programs. For example, racial tension in the local 
community or elsewhere could affect students' racial attitudes 
or achievement; a teachers ' strikri during the year could affect 
achievement, etc. However , since all students are participants 
in the desegregation program and are subject to the same outside 
influences, the effects' of desegregation and the effects of 
outside events are confounded, and are not easily separable . 
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^' 2. Type IV Designs; Longitudinal ■comparisons of score s 
^or differenc cjroups of students > In Type IV designs (Figur^ci . 
V-4) the test scores of students in a particular grade (or 
grades) are compared before and after desegregation. For 
example, scores of students in grade X before desegregation 
are compared with scores of other students in grade X following 
^ desegregation (comparison a in Fig. V-4) . It is assumed that 
changes in the test scores of different waves of students from 
pre to postdesegregation may be attributed to the desegregation 
program. 

a. Advantages of Type ly designs . Type IV designs 
are comparatively easy to administer since data for the same 
group of individuals do not n'-ed to be retained and matched 
with new data from later testings.^ Moreover, compared with 
Type II or Type III designs, less ^attention need be paid to 
the details of student assignments i to-particular classes or. 
schools, either with regard to the \impact of assignments on the 
environmental variables to be controlledV or with regard to the 
subsequent feasibility of testing students in new locations. 

From an interpretive point of view. Type. IV de- 
^ signs effectively enable the district to rule out maturational 
factors as a source of test score differences since, with grade 
(age) held constant, maturation can be assumed to be equivalent 
for- students, tested prior to desegregation and for students . 
tested following desegregation. Familiarity with the test may 



ERIC 



- 135 - 



Time 




Grades 


- 


X 


X+1 X+2 . . • 


Predesea. 














• 






Test 
A 

1 




Program 


1 

t 
1 

j (a) 




Postdeseg. 


1 




2 


1 

Test 

















Fig. V~4. A Type IV Design: 
comparison referred to in text. 



Letter in ( ) indicates 



also be eliminated as a possible source of confounding, since 
students of the same age should have taken the test an. equal 
number of time^ and should, therefore, be equally familiar with 
it. 

b. Disadvantages and problems of Type IV designs . 
With the exceptions just noted. Type IV designs share the inter- 
pretive difficulties associated with • Type III designs. In short, 
it is difficult to attribute changes in test scores specifically' 
to the desegregation program whenever: ' 

I 1) The quality of schooling has changed con- 
currently with the implementation of the desegregation program, 
e.g., the school or district has introduced smaller classes. 
Specialized staff and facilities, tutoring services, etc.; 

2) Curriculum changes have occurred (e.g., new 

reading programs, intercultural relations units, etc.) which 

i ' . ■ 

relate directly to performance on the effectiveness measures; 

3) Teacher characteristics have changed follow- 
ing desegregation, as a result of nonrandom teacher turnover 
and replacement; 

4) . The SES composition of classrooms has changed 
as a correlate of changes in racial composition; 

5) Community racial attitudes have changed as 
a consequence of contemporary social events • 

In addition to these five sources of confounding, 
another potential problem for analyzing and interpreting data 
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in Type IV designs lies in the possibility that important 
characteristics of. the school population may change during the 
course of the, study. When an entire district is desegregated, 
and evaluation of the desegregation program continues over a 
period of several years, the school population may undergo 
systematic change during that time. For example, families 
opposed to desegregated schooling for their children may move 
away, and may be replaced by other families more willing to 
accept racially heterogeneous schools. in this case, the pro- 
portion of black and white students could remain stable, bu;i^ 
the attitudes and/or achievement 'of students might change as a 
consequence of the change in parental attitudes." There is also 
the possibility that white families might leave and be replaced 
by black families, altering the proportion of black students, 
over the time period studied and thus changing tJie levels 
an important independent or control variable. Both examples 
illustrate population changes which. would make it difficult to 
interpret changes in postdesegregation test scores. 

While school district officials can do little to 
control turnover in the student body, provi«iion should be made 
for checking the comparability of the population served (e.g., 
by SES, race, etc.) over the time period studied. If it is 
found that systematic changes in the student body have taken . 
place or are in progress, the district should seek design and 
statistical assistance before . undertaking any further testing 
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or d^ta analysis 

3,. Type V Des^igns . Type V designs (Fig, V^5) merge the 

essential features of Type III and Type IV designs. They 

combine repeated testing over time of the saihe group of students 

■ ■ ' . ^ \ . ■ ■ / 

with the testing of successive waves.of students of selected 

grade level (s). Preferably,, testing for Type V designs should 

begin two years before the start of" desegregation, in order to 

establish adequate baseline^^ data for later comparisons. 

With reference to Figure V-5, comparison of successive 
diagonals for the same grades (e.g., a versus b) may be used 
to assess differences between changes in scores during a year 
before desegregation and those during a year following desegre- 
ga.tion. In this analysis, maturational effects are controlled. 
Comparison of diagonals between different grade levels (e.g., 
b versus d, with a- and c'as control data) may be used to assess 
. I the. effects on students of being desegregated at different 

ages. The effects of desegregation on students of a particular 
age can be evaluated by analyzing vertical score changes within 
grades (e.g., e versus g^) . Moreover, comparisons between verti- 
cal changes in different, grades (e.g., £ versus, h, with e and 
f^ as control data) can providfe an additional way of evaluating 
. the differential effects of.^ desegregation as a function of age. 

As more years (t3*-**'^n) added to the basic de- 

. sign shown in Fig. V-5, any factors associated with the start 
of desegregation which might either inflate or depress scores 

139 - 



ERLC 



Time 



Grades 



Predeseg . 



Program 



Postdeseg, 



X 



Test 



X+1 



X+2 



(g) 



^Test Test 

^vc; ^ (C) . 

|(e)^ 1(f) N 

Test Test Test 

4 > - • 4 ^ ' 
1 \ I \ 

I \(d) 

(h) \ 

Test Test 




\ ' 
\ 

\ 



Test 



•c 
n 



Fig. V-5. A Type V Design: Letters in ( ) indicate 
comparisons referred to in text. 



- 140 ^ 



temporarily (e.g., initial anxiety, excit-ement, confusion, etc.) 
would have time to dissipate, and thus more s;table findings 
should be obtained. As. more grades are added, groups of stu- 
dents can be followed for a longer periodof time, and it 
becomes feasible to asses s-^*' the long-term effects on students 
of. having been desegregated at different times in their school 
careers. 

a. Advantages of Type V designs . It is an advantage 
of Type V designs that longitudinal) data for a par'ticular 
group of students may be collected with maturational factors 
controlled. Moreover, depending on the care with which the 
sample is selected, it may also, be possible to control famil- 
iarity with the test for at least some of the statistical com- 
parisons . ^' 

The principal advantage of Type V designs is 
that they provide considerably more complete and more varied 
information, than Type III or Type IV designs alone. Since at 
least two ways of analyzing the data (vertically for between- 
group information and diagonally for developmental information) 
are available, it becomes possible to validate one set of 
findings against the other. If all analyses result in relative- 
ly consistent findings, the strength of any conclusions which 
may be drawn is increased. . \ 

Several other types of interesting and important 
analyses are also possible when Type V designs are used, but 
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their description is not appropriate within the scope of this 
chapter in view of their complexity^ Districts considering 
Type V designs should plan to work with a consulting statistician 
to define and examine all such possible cuialyses. 

b. Disadvantages of Type V designs . Type V designs 
are the most costly of any 6f the designs discussed in this 
chapter because of the need for repeated testing of the same 
students over an extended period of time/ because a larger 
sample size is usually required, because data collection must 
begin well. in advance of desegregation, and because the greater , 
complexity of analyses demands a higher degree of statistical 
expertise and analytic (e.g., computer) capacity. 

Despite their greater cost and complexity. Type 
"V designs may not provide wholly unambiguous results , since 
they are subject to many of the same sources of confounding 
noted above in connection with otlier designs , e .g. / changes in 
school quality, changes in teacher characteristics, changes in 
student body characteristics, simultaneous introduction of 
multiple new programs, changes in SES characteris'::ics of the 
educational environment, etc. 

Another uncontrolled source of variation in test 
scores in Type V designs stems from the fact that all children 
born in the same year constitute a unique "cohort" of individ- 
uals who always share in coiranon the experience of being a 
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particular age at a particular point in time .* Since no group, 
before or after them ever experiences exactly the same set uf 
events at the same ages as they, the cumulative life ex^rience 
of any age cohort is somewhat unique. 

In Figure V-5 each diagonal represents a different 
age cohort of students. . While successive cohorts, undoubtedly 
share many characteristics and hence are. apt to be highly simi-, 
lar at any given grade level (e.g.,.. as second graders , third 
graders, etc.), the uniqueness contributed by membership in a 
particular cohort may play a part in test data.. Thus some part 
of the difference between the effects of desegregation at dif- 
ferent ages (e.g., b versus d) may be due to membership in 
different cohorts, and may not simply be a function of age. 
The factor of cohort membership may also account for changes 
over'time (e.g., e versus ^) in the scores of successive waves 
of students at a given gradfe level, whether or not there has 
been a change in school programs.. 

Because Type V, designs provide the greatest amount 
of information when carried out^Gross several grade levels 
and when continued over a period of several years , they are 

This source of - confounding also applies to Type III and Type 
IV designs, but has been reserved for discussion here" in the 
interests of clarity of presentation. 
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best suited to districts which normally conduct an annual test- 
ing program in several or all grades. Moreover, since many 
districts do not formulate their desegregation plans more than 
one year ahead of time, the two year lead time required*- may be 
prohibitive except where districts already have prior informa- 
tion on all students. As a "general guideline, however, districts 
lacking sufficient record-keeping, data storage, and data re- 
trieval capability should probably not attempt Type V designs. 

4. The special case of grade-arvear desegregation > Some 
school districts desegregate gradually, on a grade by grade 
basis, usually beginning during the first desegregation vear 
with the first grade and desegregating one additional grade 
each year thereafter. When this pattern is followed, any 
child is either desegregated from his first grade in school 
or is not desegregated at all (at least not in elementary 
school); Accordingly when a grade-a-year plcin is employed, 
some of the types of designs previously discussed cannot be 
used. 

Neither Type II designs nor Type III designs are 
appropriate for evaluating grade-a-year desegregation plans. 
These designs require predesegregaticn and postdesegregation 
scores^, for each iiidividual, while in grade-a-year plans 
individual students will have either pre or postdesegregation 
scores, but not both. 
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Type IV and Type V designs may be used, but only in 
those grades already ^reached by desegregation • For example, 
in the first year of desegregation, program effects could be 
assessed only fc^ first graders? in the second year, they could 
be assessed for first and second graders, etc.* It would be 
six years before the. effects of desegregation could be evaluated 
in the sixth grade. Hence, the hig"her the grade levels to be 
evalua.ted/ the longer the study must continue. 

Figure V-6 shows how^ a Type V design might look when 
applied to a grade'-a-year desegregation plan. Development ...in 
Segregated environments , can be compared with development in 
desegregated environments by analyzing differences between 
successive diagonals for the samq grades (e.g,, a versus b) . 
As additional grades are desegregated, comparisons between 
vertical changes in scores can be made (e^g., d versus f, with 
£ and.e as control data) to assess. the cumulative effects at 
different ages of having been desegregated since the first year 
in school. 

5. Siimmary Table . Major cost factors associated with 
each of the designs discussed in this chapter are summarized 
in Table V--1. Every type of design has features which could. 

This restriction applies only to assessment of program 
effects on those children directly involved. "Spillover" 
effects on adjacent grades can be measured at any time. 
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Time 



Grades 



Predeseq. 



t 




Program Start 



Test (Seg . ) 



Test (Seg.) 
1(c) 

Test (Seg.) 

A \ 

I \ 
I \ ' 

\(a) I 

-"(d)* \ 1(e) 

i : \ V 

Test (Deseg. )^ Test (Seg- ) 

Test (Deseg.) 



-n 



Fig. V~6. A Type V Design under Special Case of 
Gjiade-a-Year Desegregation: Letters, in ( ) indicate 
comparisons referred to in text. 
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Table V-1 

Summary of Cost Factors in Five Types of Evaluation Designs 



Cost Factors 



Administrative Requireraents 

Need to keep track of 
individual students 

Length of time required 
for study 

Administrative complexity 

Data processing costs 

Need for . statistical . 
consultants 



Designs 



I 


II 


III 


IV 


V 


No 


Yes 


Yes 


No 


Yes 


Short 


Mod^ 


Mod 


Mod 


Long 


Low 


Mod 


High 


Mod 


High 


Low 


Mod ' 


High 


Mod 


High 


Low 


Mod 


High 


Mod 


High 



Interpretive Limitations 

Maturational confounds 

Self- selection 

Loss of individual 
students 

General change in 
student body 

Test- taking experience 
Interpretable Data Yield 



No 


No 


Yes 


Yes ' 


Yes° 


No 


No 


Yes 


Yes 


No 


No 


Yes 


No - ■ 


No 


Yes 


Low 


Mod 


Low 



No 
No 
No 

Yes 

No 

Mod 



No 
No 
Yes 

Yes 

Maybe 

High 



^ Mod = Moderate 

^ Confounding factors which potentially limit the interpretability of all 
designs have been nmi tted from this table, e.g., other new programs 
simultaneous with desegregation, changes in school or teacher quality 
associated with desegregation, etc. 

^ For IIA: No \^ 
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make it attractive to school districts as well as features which 
could make it unattractive. Each district must therefore decide 
whicli type-vOf design will be most appropriate, given the rjature 
of its own desegregation program, tha char £LCteristics of the 
student body and community it serves, and the fiscal resources 
and technical capability which are available for carrying out 
the evaluation. 
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' Appendix 
The Development of Racial Balance Indicators 
Based on the Probability of Occurrence 
of School and Classroom Distributions 
The purpose of this appendix is to bridge the gap in 
Section" III. B. 3. b of these guidelines between the probability 
of occurrence of a distribution and the measure, or function/ 



of this probability which can be taken as an indicator of 
racial balance. 

The probability of occurrence of a particular distribu- 
tion of students from minority group m to a set> of schools 
or classrooms of specified capacity can be determined using 



Let: . ^ 

N = • the total number of students to be distributed; 
N = the number of these students from minority 



t)he techniques of combinator^ial analyses . 



m 



group m; 



N 



r 



N - N = the remaining number of students; 



c 



the number of. classes or schools to which 



the students are to be assigned; 



n 



the. number of students in each class or 



c 



school (c = 1 , 2 , 3 , 



• • • / 



C) ; 



W. Feller, 0£. cit . 



n^^ = the number of students from minorJ.ty. group. 

m in class or school c; 

n^^ = n - n - the remaining number of students 
rc c mc ^ . 

in class or school c. . 

It is assumed that N, N , N , C and n are all predetermined 

m' r c 

and fixed. Note that 



C C / -I C 

S^c = ^' = % 

C=l C=l C=l 



= N . 
rc r 



Suppose that we begin by choosing students to fill the 
first classroom or school (c=i). A total of n^ students 
must be chosen from the total number of students, N. The 
number of different ways in which we can choose n^ students 

from the total population so that n are from minority 

mi ■ 

group m and /n^^' - (n^ - n^^ ) ar'e not from minority group 
m is : 



n I (N 
mi ■ m 



V I f "r! 1 , 



where, for example, N I is read . "N factorial," and is 

m* m 

defined as 



N^l = (N^) (N^ - 1) (N„ - 2) . . -3.2.1 
m ' mm m 
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Each way of choosing n and n students from the total 
. population' constitutes a different arrangement s Two arrange- 
ments are different if one or more of the individual students 
chosen in one arrangement is not present in the other. How- 
ever , each arrangement consists of the same number of 
m-students and r-students. 

We now proceed to .fill the second classroom or school by' 
choosing n m-students and n = n - n r-students from 

•m2 r 2 2 ITl2 

the remaining population of (N - n - n ) students. The 

. ^ ^ . ^ mi ri ^ 

number of different ways in v/hich this can be done is: . 

2v / \ _ I m mi ' I 

^2^%2'"r2^ ~ (n^ ) l(N„ - n . - ) ' " 
L m2 ■ m mi ma ■ J 

(N - n ) I 
r r 1 ■ 

(n ) I (N - n - n ) | 
r 2 • r r 1 r 2 • 

This process is repeated until all students in the population 
have been assigned to classrooms or schools. 

Therefore the number of different ways in which N. stu- 
dents can be assigned to C classrooms or schools so that 

.the classroom or school populations are n + n = n , n + 

- ^ mi n 1 m2 

n = n , . . . / n_ + n " = n^ , is,: 
r 2 ■ 2 mc r C C ' 
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11 A. = A . A • A • . . . 

I I ^ 12 3 



1=1 



(N^!)(N^!) 



1=1 



1 = 1 



where H denotes the continued product: 



I |x. = x*X'X •: 



1=1 



12 3 4 



and where n + n + 

12 



The number of ways in which N students can be assigned 
to C classrooms or schools so that a total of n^ are in the 
first classroom or school, a total of n^ are in the second, 
a total of n^. are in the third, etc., without regard to 
whether the students are from minority cyroup m or not , is 



^0 = -C 



N! 



rr 



(n. f) 

1 • 



1=1 



where, again, n^ + n^ + . . . = N = + N^. 

' Therefore the probability of occurrence of a particular 

distribution of N = N + N students, for which the classroom 

m r 



or school "populations are n^^ + n^.^ = n^, n^^^ + n^^ = n^. 
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n^^ + ri^^ = , can be found from: 



.T^""^ (N I)(N ,)T^ (^i!^ 



P = = ^' ^' : (A-1) 

DA C C 

(N!)TT"(n^.!) TVCn^i! ) 

i=i i=i 



Here Pj^ denotes the probabilit;,- of occurrence c f distribution 
D. D is an arbitrarily assigned label (D =. 1, ^, 3 , . .) 
used to identify, different distributions. Thi^D formula is a 
generalized hypergeometric distribution, and has been used to 
calculate the probabilities for the example in Figure III-3. 
For that example, the formula reduces to: 

Pd — (61)'- ^ . (A_2) 

The discussion in. Section III.B points out that P^^ is not 
a suitable racial balance indicator beciause its value is 
affected by factors such as school or district population 
size, and school district racial heterogeneity. In addition, ' 
the formula for P^^ is rather unwieldy. 

To. construct a measure., or function, of P^ from which 
suitable racial balance indicators can be constructed, it is 
useful to work with several examples. One convenient example 
is the set of distributions appearing in Figure III-3. This 
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set (distributions 1 through 7) is repeated. in Table A-1 
together with various quantities which will be discussed 
subsequently, in this appendix. Further -examples of distri- 
butions which will i)e^^us.!ed to illustrate this discussion 
appear in Tables A-2 and A-3. 

Finding a measure of Pj) which is independent^ of N. The 
first step is to identify some function of Pj^ which is inde-=L. 
pendant of N, the number of students being assigned. Table 
A- 2' shows the least and most probable distributions of equal 
numbers of "minority" (m) and "ma jority"% (r ) students to two 
classrooms of equal size for three different values of N: 
N = 12 (distribution 1 and 4, transcribed from Table A-l); 
N = 32 (distributions 8 and 9) ; and N = 64 (distributions 
10 and 11). Inspection of P^^ for these six distributions 
indicates that.: 

• P corresponding to the least probable distribution 
(maximum imbalance) falls off very rapidly as N in- 
creases. For N = 64, (P^) . = 4.88 x lO^VS 

D min 

• The values of (P^^) . become exceedingly small as N 

D' min ^ 

increases ; ^ ' 

• Pj^ corresponding to the most probable distribution 
(maximiom balance) also falls off with increasing N. 

The precise way in which depends on N is difficult 
to find. In many problems of this kind, it is useful to 
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work with the logarithm of P^^, rather than P^^ itself, in an 
attempt to determine the dependence of Pj^ on N. 

The logarithm (to the base 2) of P^^ for each of the six 
distributions in Table A-2 is also shown in the table. In- 
spection of the values of log2^b ^^^^^^P^^^^^? least 
probable distributions for the three values of N appearing 
in Table A-2 shows that they are all fairly close to N in 
magnitude. This pattern suggests that the quantity (log^P^) /N 
(or rather, to get rid of the inconvenient minus sign, ' 
(-log2Pj^) /N) be examined to see how .clos^ it comes to having 
the required property. 

Values of (-log2Pj^) /N f or the six distributions in Table 
A-2 are also entered in the table^r^- Inspection of these values 
indicates that: 

• The value of (-log2pQ)/N corresponding to 

tends towards one as N increases ; — 

• The value of (-log^P^)/N corresponding to (P-,) ^^^^^ 

^2 D ^. ^. D max 

tends towards zero as N increases. 
Thus the quantit^^ (-log2PQ)/N appears to show much less vari- 
ation with N than Pj^ itself. . ^ 

The last step is to write the expression for P^^ (equation 

A~l) *in a form which permit^ capitalizing on the observations 

\ 

made so far. 5*0 do this, a well-known approximation to the 
factorial , (Stirling' s formula) can be used: 
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x! ^ /27rx x^e ^. 

Replacing ^qill the factorials in equation (A-1) by this formula, 
taking the logarithm, dividing by and multiplying by (-1) , 
we obtain: 



N 



^ T + (other terms) 



(A- 3) 



where : 



T = log2N - ^ 



1=X 



- (n .log^n . + n . log^n .) 
, mi ^2 mx ri ^2 rx 



1=1 



(A- 4) 



The quantity T for the six distributions shown in Table A-2 
appears in the last column of the table. The range of T is 
zero to one for each of the three values of N Appearing in 
the table. T will be examined to see if it meets the re- 
quirements placed on acceptable measures of P . 



The quantity T can also be obtained from information theory. 
Information theory leads to the development of uncertainty 
statistics, which represent an alternative, but less direct, 
way of describing student distributions. In the language of 
information theory, T is called the contingent uncertainty- 
It measures the extent to which uncertainty^^bout a particular 
student ' s race is reduced by knowledge of the^articular 
classroom or school to which the student is assigned^^^^f 
schools or classrooms are maximally balanced, the uncertain:ty 
about a student's race is undiminished by knowledge of that 
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The quantity T can be shown to be independent of N for 
any distribution of students to schools or classrooms. Sup- 
pose we are dealing with a particular distribution, and sup- 
pose N is changed to yN (where y is some constant) without 
changing the proportions of students from different groups 
in the population and without changing the number of 
schools or classrooms. (These conditions require the school 
or classroom capacities to change from n^ to yn^.) Replacing 

N. N , N , n,, n . and n . in equation A-4 with y 'times each . 
r m 1 mi ri 

quantity, we obtain: 

T-(YN, yN^, yN^, . . .) - T(N, N^, , . . .). - 

Table A-3 illustrates the independence of T on N. Three 
different values of N are considered. The racial heteroge- 
neities are the same for each N (33.3%). For each value of 

T is calculated for three different distributions : an 
intermediate distribution (distributions 1 2\ 15 and 18); the 



student's school or classroom assignment, i.e., T = T^in = 0 . 
If schools or classrooms are maximally unbalanced, the uncer- 
tainty about a student's race is greatly reduced by knowledge 
of that student's school or classroom assignment, i.e., 
T = Tmax. - . ' 

Discussions of information theory and its applications can 
be found in: F. Attneave ^ Application of Information Theory 
to Psychology , Holt-Dryden, New York, 1959; and W. R. Garner, 
Uncertainty and Structure as Fsychological Concepts , John 
Wiley & Sons, Inc., New York r 196;2. ^ 
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least probable - distribution (13^ 16 and 19); and the most 
probable- distribution (14^ 17 ^nd 20). Each intermediate 
distribution shows the same extent racial balance, ahd 
the calculated value of T is the same for each- (0,073) al- 
though the value of N is different for .each. The values of 
T^^^ (correspond.ing to the least probable distribution for 
each N) are the same for each N (0,379>r and the value of 
'^min ^^^^^^^spo^^i^^^o "tLhe mp^t probable distribution) for 



N = 54 is 0.000. 



/ 



The. values of T . for N = 36 and N = 72 are almost .zero 
mm 

(0.001)% but not quite. The reason is that the class sizes 

do not permit minority students to be distributed in such 

a way as to reproduce in the classrooms the proportions of ^ 

minority students in the populations of 36 and. ,72 students 

being assigned. The distributions closest to perfect balance 

in these cases are dirjtributions 14 and 20. In both cases , 

the proportions of m-students are 0,35 in classroom 1 and 

0.31 in classroom 2, compared to 0;33 in the student popula- 

tions as a whole.- Consequently , T . is not exactly zero. In 

^ mm ft 

practice, such situations do not affect the interpretation of 
racial balance indicators when the number of pbss4.ble dis- 
tributions of minority students to classrooms or schools is . 
sufficiently large. ^ 



Distributions involving small numbers of minority students 
are discussed in Chapter III. 
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In addition to being^lln^ependent of N, the variation of 
T with is also consistent with the operational definitions 
of racial balance^ Racial balance is the extent to which the 
proportions of m-students in schools or classrooms approach 
the proportion of m-students in the student populatiph be- 
ing assigned.. It can be shown formally that T assumes its 
minimam value (zero, or very close to it as in the above 
examples) for the . distribution corresponding to maximum racial 
balance, i.e., when the proportions of m-students in schools 
or classrooms are as close as possible to the proportion of 
m--students in the student population being assigned • Fur- ^ 
thermore, the minimum value of T will be zero, or nearly 
zero, regardless of the racial heterogeneity of the student 

population (e.g., compare distributions 1, Table A-1, and 

^ - . ■■ • 

17, Table A-3) . However, the maximum value of .T (correspond- 
ing to maximum racial imbalance) does depend on the racial 
heterogeneity of the student population. This problem will 
be dealt with in the next part of this appendix. 

Finally, although T is not equal to {-log^^^) /l^i , or to 
any other function of P^^ which can be simply expressed, it 
appears to vary "monotonically " with P^^. That is, the 
ordering of values of P^^ corresponding to different distri- 
butions (frp*T^ higher to lower) is not affected by dropping 



Using Lagrange's method of undetermined multipliers. 



the "other terms" in equation A-3. This monotonia property 

is all that we require of a measure of P^^ in order to use it, 

in the manner described, in Chapter III , to construct suitable 

racial balance indicators. 

Eliminating the dependence of T on racial heterogeneity . 

The value of T^^^, corresponding to maximum racial imbalance, 

is only equal to one if the racial heterogeneity of the stu- 

dent .population is 50%. if the heterogeneity is not 50%, 

T will be less than one. For example, in Table A-3, the 
max 

heterogeneity is 3 3% and T 0.379. Furthermore, T 

max . \ ■ . ' max 

will be different for different heterogeneities. 

This difficulty can be overcome by using a linear trans- 
formation of T, , rather than T itself, as an indicator of ' 
racial balance. The particular linear transformation used 
in Chapter III is « / 



B = 1 00 X 



T - T 
max 



T - T . 
max min 



The quantity B is in the form used in Chapter III t/o define 
both school and classroom balance indicatorsV The^ indicator 
B has all the desirable properties of ^ which were discussed 



If — ^ — 

The magnitude of T, an^ hence Tj^ak/ also depends on the 
base of the logarithms used. The statement LsJtt^\i^ only for 
logarithms to the base 2. However, B, is independent of the 
base chosen for the logarithms- ^ \ 
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earlier "in the appendix and, in addition, takes; on the same 
value (i.e., zero) for the least probable distribution of a 
student population of any racial heterogeneity. 
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Distribution of 6 m-students (from minority group mL 
and 6 r students to two classes, six "^^^ 
students to each class 
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32 


0 


4 
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-60. 8 31 


0.950 
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11 


64 


16 


16 


16 


16 




0.197 


- 2.344 


0.037 


0 



Most and least probable distributions of 12, 32 and 64 
students to two classrooms of equal capacity. 
Racial heterogeneity is 50% . 
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Table A- 3 
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36 


12 


8 


0 


16 


0.379 


(max) 


14 


36 


7 


13 


5 


11 


0,001 


'(min) 
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54 


10 


20 


8 


16 


0 .000 


(min) 


18 
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72 


24 


16 
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72 


14 


26 


16 
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0.001 - 
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ma 
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Most probable, least probable and intermediate distributions 
of 36, 54 and 72 students to two classrooms 
of unequal capacity. 
Racial heterogeneity is 33%. 
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